Tag Archives: software

SSD

Posted on by
Reply

When I built Whisky, my current work-a-day desktop, back in November 2009 I wanted to boot from one of those blazin’ solid-state drives. Bummer, though, either they were seriously expensive or performed poorly. Poorly, of course, was a relative term; for the most part even the poorest smoke conventional hard drives. Still, as the build expenses mounted the SSD finally fell off the spec list.

Sometime after the build, Seagate brought their hybrid drives to market. Hybrids combine a conventional spinning disk and conventional cache with a few gigabytes of SLC NAND memory configured as a small SSD. The system sees the drive as it would any other drive; an Adaptive Memory (Seagate proprietary) algorithm monitors data use and keeps frequently used stuff on the SSD. You’ll find people arguing over whether or not a hybrid drive provides any kind of performance boost. I wrote about my experiences with the Seagate Momentus XT (ST95005620AS) back in June 2010. Today when I build a multiple drive system I routinely spec a hybrid as a boot drive. It’s cheap and it helps.

Corsair Force Series GT CSSD-F240GBGT-BKSo about a month ago I ran across a good deal on a fast SSD, a Corsair Force Series GT (CSSD-F240GBGT-BK) and I jumped on it. The specs are just tits: sequential reads and writes of 555 and 525 MB/s respectively. (Sure, that was with a SATA 3 interface and my motherboard only supports SATA 2; I wouldn’t see numbers like that, but still… It even looks great.

Integrating the thing into a working system was a bit of a challenge, mostly because I didn’t want to purchase additional software simply to clone the existing boot drive. I’ve got no trouble paying for software I use; it simply seemed like too much for something to be used but once. So part of the challenge was to find a cost-free alternative.

Strategy and Concerns

The general strategy would be to clone the current two-partition boot drive to the SSD, swap it in and enjoy the performance boost. The SSD partitions would need to be aligned, of course, and somewhere along the way the C partition would need to shrink to fit onto the smaller SSD.

The top concerns came down to security and reliability. Erasing a conventional hard drive is easy: repeatedly write random data to each block. You can’t do that with SSDs. Their blocks have a specific (and comparatively short) lifetime and so on-board wear-leveling routines become important. When data is overwritten, for example, the drive writes the data elsewhere and marks the old blocks for reuse. And unlike conventional drives, it’s not enough to simply write over a block marked for reuse; the entire block must first be erased. The bottom line is you can’t ever know with certainty whether or not a SSD is ever clear of confidential data. Disposing of them securely, then, means total destruction.

As for reliability, a conventional hard drive has to have some pretty serious problems before it becomes impossible to recover at least some data. There’s generally a bit of warning – they get noisy, start throwing errors, or something else that you notice – before they fail completely. Most often an SSD will simply fail. From working to not, just like that. And when that happens there’s not much to be done. This makes the issue of backups a little more thorny. If it contained confidential data at the time of failure you’ve got a hard choice to make: eat the cost and destroy the device, or RMA it back to the manufacturer (losing control of your data).

Considering backups, you can see that monolithic backups aren’t the best solution because they’re outdated as soon as they’re written. Instead, a continuous backup application, one that notices and writes changed files, with versioning, seems prudent.

In my case, this is to be a Windows 7 boot drive and and all confidential user data is already on other storage. The Force Series GT drive has a 2,000,000 hour MTBF, fairly high.

Software

SSDs are fast but they’re relatively small. It’s almost certain that existing boot partitions will be too big to fit and mine is no exception. Windows 7 Disk Manager will allow you to resize partitions if the conditions on those partitions are exactly right. There are commercial programs that will do the job where Windows won’t but my favorite is MiniTool Partition Wizard. I didn’t really want to do that in this instance. The fundamental problem I had with pre-shrinking is that it would involve mucking with a nicely working system. Come trouble, I wanted to simply pop my original drive back in the system, boot and get back to work.

For cloning and shrinking partitions there are several free or almost free applications. I found that most of them have drawbacks of one sort or another. I’ve used Acronis before – Acronis supplies OEM versions of their True Image software to some drive manufacturers, it’s an excellent product. But their free product won’t resize a partition image, bummer. I used EaseUS some years back, too, but a bad run-in once with their “rescue media” – in that case a bootable USB stick. My disks got hosed pretty bad from simply booting the thing and I… wasn’t pleased. Maybe they’ve gotten better, people say good things about ‘em, but I wasn’t confident… Paragon seemed very highly rated but in testing I had too many validation failures with their images. Apparently the current version is worse than the back revs. Whatever, I was still uneasy. I ended up settling on Macrium Reflect from Paramount Software UK Ltd. For no rational reason the name of this product bothered me, sending it to the bottom of the test list. Macrium. The word makes me think of death by fire. I was reluctant to even install it. About the only negative think I’ve got to say about Macrium is that it takes a fair bit of effort to build the ‘rescue disk’ – bootable media to allow you to rebuild a failed boot volume from your backup image(s). The rescue media builder downloads and installs, from a Microsoft site, the Windows Automated Installation Kit. WAIK weighs in at more than 2 GB. The end result is a small ISO from which you can make bootable media of your choice. Except for that final burn – you’re on your own for that – the process is mostly automated; it just takes a while. Probably has to do with licensing or something.

Finally, I bought a copy of Genie Timeline Pro to provide the day-to-day realtime backup insurance, mentioned earlier, that I wanted.

Preparation for Migration

I started by installing both Gene Timeline Pro and Macrium Reflect and familiarized myself with each. I built the rescue media for each, booted from the media, and restored stuff to a spare drive in order to test. It’s an important step that many omit, but a backup that doesn’t work, for whatever reason, is worse than no backup at all.

I did some additional maintenance and configuration which would affect the C: partition. I disabled indexing and shrunk the page file to 2GB. The box has 8GB RAM and never pages. I suppose I could omit the page file entirely, but a warning is better than a BSOD for failure to page. I got rid of all the temp junk and performed the usual tune-up steps that Windows continues to need from time to time.

Satisfied, I imaged the System Reserved partition and the C: partition of my boot volume, verifying the images afterward. For each partition, which I backed up with separate operations, I used the Advanced Settings in Macrium Reflect to make an Intelligent Sector copy. This means that unused sectors aren’t copied, effectively shrinking the images. Then I installed the SSD via an eSATA port. Yes, this meant it would run even slower than SATA 2 but it saved a trip inside the box.

It was at this step that I noticed the only negative thing about this drive. The SATA cable is a bit of a loose fit. It doesn’t accept a retaining clip, if your cable is so equipped. Ensure there’s no tension on a cable that might dislodge it.

Creating Aligned Partitions

Partition alignment is important on SSDs both for performance and long life. Because of the way they work, most will read and write 4K pages. A very simplistic explanation is that when a partition is not aligned on a 4K boundary, most writes will require two pages rather than one which decreases performance dramatically and wears the memory faster. (There’s more to it than that, really, but you can seek that out on your own. The Web’s a great teacher. Being the curious sort I learned more than I needed to.)  Windows 7, when IPLed, will notice the SSD and build correctly aligned partitions for you. Some commercial disk cloning software will handle it automatically, too. But migrating users are on their own. Incidentally, it’s theoretically possible to adjust partition alignment on the fly, but if you think about the logistics of how this might be done – shifting an entire partition this way or that by some number of 512 byte blocks to a 4K boundary – you’ll realize it’s more trouble than it’s worth. Better to simply get it right in the first place.

Fortunately it’s easy!

Using an elevated command prompt (or, in my case, a PowerShell), use DISKPART. In my case, my existing System Reserved partition was 71 MB and change, and the remainder of the SSD would become my C: partition.

diskpart
list disk
select disk <n> (where <n>is the disk number of the SSD)
create partition primary size=72 align=1024
active (the System Reserved partition needs to be Active)
create partition primary align=1024 (no size specification means use the remaining available space)
exit

You can also use DISKPART to check the alignment. I’ll use mine as an example.

diskpart
list disk
select disk <n> (where <n>is the disk number of the SSD)
list partition
exit

My partition list looks like this.

Partition ### Type             Size    Offset
------------- ---------------- ------- -------
Partition 1   Primary           70 MB 1024 KB
Partition 2   Primary          223 GB   73 MB

To check the alignment, divide the figure in the Offset column, expressed in kilobytes, by 4. If it divides evenly then it’s aligned. For Partition 1, the System Reserved partition, 1024 / 4 = 256, so it’s good. Partition 2′s Offset is expressed in megabytes so we have to convert to kilobytes first by multiplying it by 1024. So, 73 * 1024 = 74752 and 74752 / 4 = 18688, so it’s good, too.

Whew!

It’s worth noting that what DISKPART didn’t show in the list is the tiny unused space – about 2MB in my case – between Partition 1 and Partition 2 which facilitated alignment.

Someone pointed out to me that partition alignment can be checked without DISKPART. Fire up msinfo32. Expand Components, then expand Storage, then select Disks. Find the drive in question and divide the Partition Starting Offset fields by 4096. If it divides evenly you’re all set!

Migration

I used Macrium Reflect to restore the partition images I created earlier. Rather than allowing the software to create the partitions (which would negate our alignment effort) I pointed it to each target partition in turn. When the restore was finished I shut the system down.

I pulled the SSD from the eSATA port and pulled the existing boot drive from the system. I mounted the SSD in place of the old boot drive. (Windows gets upset when it finds multiple boot drives at startup, so it’s a good idea to have just one.) I took extra care with the data cable.

I powered up and entered the system BIOS, walked through the settings applicable to a drive change, saved and booted.  Things looked good.

Living With the SSD

Wow! Coldstarts are fast. (See below.) So fast that getting through the BIOS has become the perceived bottleneck. Applications start like lightning, especially the first time, before Windows caches them. Shutdowns are snappy, too. (See below.) There’s no shortage of anecdotes and benchmarks on the ‘net and I’m sure you’ve seen them. It’s all delightfully true.

But all wasn’t perfect. After a week or two some new patterns seemed to be emerging.

Every so often, unexpectedly, the system would become unresponsive with the drive use LED full-on solid, for some tens of seconds. Most of the time the system would return to normal operation but depending on what application was doing what at the time, the period of unresponsiveness could sometimes cause a crash. Sometimes the crash would be severe enough to bring on a BSOD. The biggest problem I have with BSODs or other hard crashes is that it causes the mirrored terabyte data drives to resync, and that takes a while. Usually the System Log would show Event ID 11 entries like this associated with the event:

The driver detected a controller error on \Device\Ide\IdePort6.

And once, following a BSOD, the boot drive was invisible to the BIOS at restart! A hard power cycle made it visible again and Whisky booted normally, as though nothing abnormal had ever occurred.

Hard to say for sure, but it seemed as though these oddities were happening with increasing frequency.

Firmware Update

Prowling the ‘net I found others reporting similar problems. What’s more, Corsair was on the case and had a fresh firmware update! The update process, they claimed, was supposed to preserve data. I checked my live backup and made new partition images anyway. The drive firmware update itself went exactly as described, took but seconds and left the data intact. The next boot had Windows installing new (or maybe just reinstalling?) device drivers for the drive, which then called for another boot. All this booting used to be a pain in the ass but when the box boots in seconds you tend to not mind that much.

Benchmark performance after the update was improved, but only marginally – nothing I’d actually notice. The troublesome hangs I mentioned seem to occur on bootup now, when they occur at all. They seem less ‘dangerous’ because they don’t interrupt work in progress at that time. So far, anyway, I just wait out the length boot and log in, followed by a cold shutdown. The next coldstart invariably goes normally, that is, very, very fast.

What’s going on? Maybe some periodic housekeeping going on in the drive? Maybe some housekeeping that was underway when I interrupted with a shutdown? Or maybe it’s that data cable? Remember, I mentioned it’s sort of a loose fit without a retainer clip. Time will tell.

Videos

I goes without saying that SSDs are fast. Many people like to judge that by how fast Windows loads. I threw together a couple of videos to illustrate.

System Startup with SSD
00.00 - Sequence start
01.30 - Power on
04.06 - Hardware initialization
13.20 - Video signal to monitors
15.83 - BIOS
23.93 - Windows Startup
39.83 - Login prompt
44.93 - Password entry complete
54.50 - Ready to work

Power on to Windows startup duration is 22.63 seconds.
Windows startup to login prompt duration is 15.90 seconds.
Password entry to ready-to-work duration is 9.57 seconds.

 

System Shutdown with SSD

00:00:00 - Sequence start
00:08.32 - Shutdown initiated
00:24.27 - Shutdown complete

Shutdown initiation to power off duration: 15.95 seconds.

 

iPad

Posted on by
3

People that know me know that I’m not a big Mac fan. By extension, not a big Apple fan either. That’s why people that know me are astonished when they learn that there’s an iPad in my house. The initial shock gives way to questions so I figured I’d just handle some of them here.

My friend Will, just the other day over on Google+, said “Trims atas advise nya.” Oh, wait a minute. iPad2That’s spam from some shitstain with an anonymous gmail account. Will actually said “Rick, what do you use it for? On TV people are watching videos, email or looking at pictures on it – nothing very interesting. Is it a glorified internet appliance?”

Well, it’s a funny thing. Tablets have been the Next Big Thing for a while and everyone has been bringing them to market. For most, er, scratch that, for everyone except Apple, success in the tablet space has been varied. For Apple success has been astounding. Eventually, I figured, we’d have to get one to play around with, to see what all the hype was about.

I think it started with a TV commercial. I casually said to Pam, “So maybe you want one of those?” and she said she wouldn’t mind. So a few days later I drank some Kool-Aid…

I’ve gotta admit, the iPad’s an absolute marvel of design and engineering. It feels really good in your hand, looks really great to your eye (both the display and the form-factor), and the UI is slick and responsive. Besides the device there’s not much in the box: a cable and charger cube (which promptly got lost for weeks) and a cute little Apple sticker. I powered it up, answered a few questions, and in a minute or two I was exploring the built-in apps. Apps. I was playin’ with apps. I felt so… trendy. We picked up the Smart Cover a day or two later. It, too, is a product of incredible thought and design. Just as you hold it near, wondering how it attaches, it attaches itself magnetically, in perfect alignment. Forty bucks.

Getting the iPad onto my network was a bit harder. We have two active WiFi networks in the house. Each serves different purpose and both are reasonably secure. (Hold your comments about being neighborly and running an open hotspot; I don’t care and I’ll only ignore you.) So I cleared the way for the iPad and tried and tried to get authenticated. Didn’t work. A search turned up plenty of others with similar problems. I forget exactly which magic incantation did the trick but after a while it was working. And here’s the thing: other than that initial hurdle the iPad connects and makes itself ready to communicate the moment you pick it up. The secret? It keeps a periodic chatter going with the router or access point, all the time. It’s always ready.

Instant-on network performance like that is usually a battery suck but Apple seems to have nailed the power management. Battery life is several weeks to a month.

“Huh? Did you say a month? Don’t you use it?”

Yup, that’s what I said: a month. And, mostly, nope, we don’t really use it all that much. None of us do. Three different people with three widely varying sets of interests and the iPad hasn’t become relevant to any of us. WTF.

What I sought most from such a device was simple (and, I might add, completely satisfied by my old netbook). I wanted to read, mostly stuff from my network where I keep a fair library of subscription material. I wanted to write, notes, posts like this, etc. And I wanted to be able to control different parts of my network, logging into a Linux console, adjusting this or that, maybe a bit of ftp to import or export a file or two, maybe shutting things down during an extended power failure.

Producing written material with the virtual keyboard is an exercise in futility. I’m not the best keyboardist in the first place but my meager productivity dropped like a stone. Y’know how they say to use strong passwords for stuff? Let me tell you, the way you need to switch modes for numbers, caps, punctuation, and everything else will have you setting your passwords to ‘asd123′ – and wishing you could skip the digits altogether – in no time flat. Forget writing.

On to reading. Well, this is actually pretty good. The display is nice, like I said. Consuming some written matter – WIRED comes to mind – the content designed for this device is, in some ways, superior to the print experience. You miss out on the tactile enjoyment of well-laid-out pulp – the color, the rich fonts – but the ease of navigation (no continued on page 134) and embedded multimedia could be a valid trade. Sometimes, at least. I mentioned that I have a rather large cache of subscription material – professional publications, books, newsletters, etc. – on a server here. The vast majority is in PDF format of one type or another. Reading any of those makes for a pretty good experience. The iPad will try to add them into the built-in iBooks app, which simply means that they’re downloaded and stored locally for use off-network.

Next up, handling network chores. Nope, can’t do that. Maybe buying a terminal app would fix that, maybe not. I’m not pressing because I have other alternatives. Also, you can’t get files onto or off of the iPad. In fact, the very concept of files on the iPad seems profoundly foreign. I’ll bet a dollar Apple would call that a feature.

Now, Pam’s expectations are markedly different from mine. She’ll play a few games, use Google+ and – gasp – Facebook, and use the Web browser. She’s bought a few apps. Sorry, can’t tell you which ones. Since the iPad is hers, it’s tied to her computer and it synced with her iTunes library painlessly and quickly. I can tell you that the Google+ client, while touted as made for the iPad, is simply an iPhone app that lives in the middle of the screen. Sizing it for the larger screen looks chunky and childish. When I tried, Hangouts didn’t work at all. Sort of too bad, that, as the hardware seems like it’d be perfectly suited to video conferencing. YouTube videos play nicely, but content-rich sites that don’t offer Flash alternatives fail.

I expected Damian to play with the iPad but he doesn’t. Not at all. Some weeks after it had been floating around in such obvious places like the dinner table, he said “Oh? We have an iPad now?” That was that. I don’t think he’s touched it since. That was a little unexpected since I think he’s in the target demographic. Oh well.

I’ve got a few closing random thoughts… The lack of multitasking hurts. The instant-on, instantly-connected Web browser – albeit a weak one like Safari – is a definite win. The lack of Flash can sometimes make a Web site unusable. Not that I’m arguing for that insecure wart on the side that is Flash, but some sites, well, that’s what they do. Sort of the way a site might be built for IE and render poorly on a standards-compliant browser. You can wish for a long time that it weren’t so. The security model kinda blows. I wouldn’t store any confidential stuff on the device. The virtual keyboard encourages the use of weak, easy-to-use passwords because good ones are such a pain to type, yet even routine updates prompt for the Apple account password.

The bottom line? I guess all told I spent something under $800 for the device, a cover and some apps. Worth it? For design, lots of points. For usefulness, very few points. Did I learn some stuff? Undoubtedly. Do I feel trendy? No, I feel like I threw away a wad of cash.

If I knew then what I know now, would I buy an iPad? No.

[edited 29 October to include this unique use for the device.]

Today is World IPv6 Day!

Posted on by
Reply

Internet Society – World IPv6 Day

How are you faring? Here, I found that we were offline when I tried to log in this morning. We’d been down for a while, apparently, as the servers had stopped their incessant chatter to my inbox. Power cycling the cable modem put things right.

Coincidence? [shrug]

Alas, Optimum Online doesn’t support IPv6. I hear they’re not alone.

Automattic in the News Today

Posted on by
1

Seen on DarkReading.com:

WordPress, the popular blog-hosting site, is reporting a breach of several of its servers.

Automattic, the company that drives WordPress, as well as Akismet, “had a low-level (root) break-in to several of our servers, and potentially anything on those servers could have been revealed,” said WordPress [...]

Folks that use WordPress or other Automattic products will want to keep an eye on this.

http://www.darkreading.com/security-monitoring/167901086/security/attacks-breaches/229401553/wordpress-reports-multiserver-breach.html

Communicating With The Outside World

Posted on by
Reply

I recently set out to upgrade a virtual host server from VMware Server to Oracle’s VirtualBox. The upgrade was a huge success. This is one of several articles where I talk about various aspects of that upgrade, hopefully helping others along the way. You might want to go back and read the introductory article Virtualization Revisited. Added 5-May-2011: Originally written using Ubuntu Server 10.04, this configuration also works without change on Ubuntu Server 11.04.

One of the things that I wanted from the new VM host was alerts for anomalous situations. Manually polling for trouble begins as a noble effort but trust me – after a while you’ll stop looking. About a year ago I was almost caught by a failing hard drive in a RAID array. Even after that incident, within a month or two I had pretty much stopped paying regular attention.

While setting up monitor/alert mechanisms on an old Windows server is quite the pain in the ass it’s a snap on Linux. Delivery of alerts and status reports via email is just perfect for me. All I wanted was the ability to have the system generate SMTP traffic; no messages would ever be received by the system. To prepare for that I set up a send-only email account to use the SMTP server on one of my domains solely for the VM host’s use as a mail relay. Then I got on with configuring Postfix, the standard Ubuntu mailer – one of several excellent sendmail alternatives.

Now maybe I’m just a dummy, but I found various aspects of the Postfix and related configurations to be a little tricky. Hence this article, which details what worked for me – and should work for you, too.

(In the stuff that follows, my example machine is named foo and it’s on an internal TLD called wan. My example machine’s system administrator account is sysadmin. My SMTP server is on mail.example.com listening on port 1212. The SMTP account is username with a password of yourpassword.)

Getting Started – Basic Configuration

Begin by installing Postfix, as you would any package.

$ sudo apt-get install postfix

For now, just hit Enter through the install questions. We’ll configure it properly following the install. You’ll be asked for the general type of mail configuration and Internet Site will be the default. Accept that by pressing Enter. You’ll be asked for the System mail name and something will probably be pre-filled. Accept that, too.

Now, go back and do a proper basic configuration.

$ sudo dpkg-reconfigure postfix

Several questions will follow. Here’s how to respond.

For the general type of mail configuration choose Internet Site.

Set the domain name for the machine. The panel provides a good explanation of what’s needed here, and chances are good that it’s pre-filled correctly. By example, foo.wan.

Provide the username of the system administrator. The panel provides a good explanation of what’s needed here. Use the name of the account that you specified when you installed Ubuntu. By example, sysadmin.

Provide a list of domains for which the machine should consider itself the final destination. The panel provides an OK explanation and it’s probably already pre-filled more-or-less correctly. But look carefully at the list that appears in the panel and edit it if it has obvious errors like extra commas. Again, using my example machine, a list like this is appropriate:

foo.wan, localhost.wan, localhost

You’ll be asked whether or not to force synchronous updates on the mail queue. Answer No, which is likely the default.

Next, specify the network blocks for which the host should relay mail. This entry is pre-filled based on the connected subnets. Unless you’ll be using an external SMTP server that requires it, you can simply remove all of the IPv6 stuff that appears here, leaving only the IPv4 entry which will probably look something like this:

127.0.0.0/8

Specify the mailbox size limit. The default is zero, meaning no limit. Accept that. Remember, all we’re planning to do is send mail, not receive it.

Set the character used to define a local address extension. The default is +. Accept it.

Choose the Internet protocols to use. Again, keeping with our earlier IPv4 decision select ipv4 from the list and accept it.

That’s it for the basic Postfix configuration! Next you’ll configure Postfix to do SMTP AUTH using SASL (saslauthd).

SMTP AUTH using SASL (saslauthd)

Since there are several commands to issue as root it’s convenient to sudo yourself as root to save some typing. Good practice dictates you should logout the root account just as soon as you’re finished.

Be careful. In this list of commands there is one – it sets smtpd_recipient_restrictions – that is quite long and may have wrapped on your display. Be sure to issue the entire command.

$ sudo -i
# postconf -e 'smtpd_sasl_local_domain ='
# postconf -e 'smtpd_sasl_auth_enable = yes'
# postconf -e 'smtpd_sasl_security_options = noanonymous'
# postconf -e 'broken_sasl_auth_clients = yes'
# postconf -e 'smtpd_recipient_restrictions = permit_sasl_authenticated,permit_mynetworks,reject_unauth_destination'
# postconf -e 'inet_interfaces = all'
# echo 'pwcheck_method: saslauthd' >> /etc/postfix/sasl/smtpd.conf
# echo 'mech_list: plain login' >> /etc/postfix/sasl/smtpd.conf

Then press ctrl-D to logout the root account.

The next step is to configure the digital certificate for TLS.

Configure the Digital Certificate for TLS

Some of the commands that follow will ask questions. Follow these instructions and answer appropriately, modifying your answers to suit your situation. As earlier, sudo yourself to root and logout from root when finished.

$ sudo -i
# openssl genrsa -des3 -rand /etc/hosts -out smtpd.key 1024

You’ll be asked for the smtpd.key passphrase. Enter one and remember it. You’ll need to type it twice, as is customary when creating credentials. Then continue.

# chmod 600 smtpd.key
# openssl req -new -key smtpd.key -out smtpd.csr

You’ll be asked for your smtpd.key passphrase. Enter it.

Next you’ll be asked a series of questions that will make up a Distinguished Name, which is incorporated into your certificate. There’s much you can leave blank by answering with a period only. Here’s a sample set of responses (underlined) based on my US location and example system.

Country Name (2 letter code) [AU]:US
State or Province Name (full name) [Some-State]:Texas
Locality Name (eg, city) []:.
Organization Name (eg, company) [Internet Widgits Pty Ltd]:.
Organizational Unit Name (eg, section) []:.
Common Name (eg, YOUR name) []:Rick
Email Address []:sysadmin@foo.wan
A challenge password []:some-challenge-password
An optional company name []:.

Then continue.

# openssl x509 -req -days 3650 -in smtpd.csr -signkey smtpd.key -out smtpd.crt

You’ll be prompted for your smtpd.key passphrase. Enter it.

Then continue.

# openssl rsa -in smtpd.key -out smtpd.key.unencrypted

You’ll be prompted for your smtpd.key passphrase. Enter it.

Then continue.

# mv -f smtpd.key.unencrypted smtpd.key
# openssl req -new -x509 -extensions v3_ca -keyout cakey.pem -out cacert.pem -days 3650

You’ll be asked for a PEM passphrase. Enter one and remember it. You’ll need to type it twice, as is customary when creating credentials.
Like earlier, you’ll be asked a series of questions that will make up a Distinguished Name, which is incorporated into your certificate. There’s much you can leave blank by answering with a period only. Here’s a sample set of responses (underlined) based on my US location and example system.

Country Name (2 letter code) [AU]:US
State or Province Name (full name) [Some-State]:Texas
Locality Name (eg, city) []:.
Organization Name (eg, company) [Internet Widgits Pty Ltd]:.
Organizational Unit Name (eg, section) []:.
Common Name (eg, YOUR name) []:Rick
Email Address []:sysadmin@foo.wan

Next, issue the remaining commands.

# mv smtpd.key /etc/ssl/private/
# mv smtpd.crt /etc/ssl/certs/
# mv cakey.pem /etc/ssl/private/
# mv cacert.pem /etc/ssl/certs/

Then press ctrl-D to logout the root account.

Whew! We’ll continue by configuring Posfix to do TLS encryption for both incoming and outgoing mail (even though we’re only planning on sending mail at this point).

Configure Postfix to Do TLS Encryption

As earlier, sudo yourself to root and logout from root when finished.

$ sudo -i
# postconf -e 'smtpd_tls_auth_only = no'
# postconf -e 'smtp_use_tls = yes'
# postconf -e 'smtpd_use_tls = yes'
# postconf -e 'smtp_tls_note_starttls_offer = yes'
# postconf -e 'smtpd_tls_key_file = /etc/ssl/private/smtpd.key'
# postconf -e 'smtpd_tls_cert_file = /etc/ssl/certs/smtpd.crt'
# postconf -e 'smtpd_tls_CAfile = /etc/ssl/certs/cacert.pem'
# postconf -e 'smtpd_tls_loglevel = 1'
# postconf -e 'smtpd_tls_received_header = yes'
# postconf -e 'smtpd_tls_session_cache_timeout = 3600s'
# postconf -e 'tls_random_source = dev:/dev/urandom'

This next configuration command sets the host name, and this one uses my example machine’s host name. You should use your own instead.

# postconf -e 'myhostname = foo.wan'

Then press ctrl-D to logout the root account.

The postfix initial configuration is complete. Run the following command to start the Postfix daemon:

$ sudo /etc/init.d/postfix start

The Postfix daemon is now installed, configured and runing. Postfix supports SMTP AUTH as defined in RFC2554. It is based on SASL. It is still necessary to set up SASL authentication before you can use SMTP.

Setting Up SASL Authentication

The libsasl2-2 package is most likely already installed. If you’re not sure and want to try to install it you can, no harm will occur. Otherwise skip this command and simply continue.

$ sudo apt-get install libsasl2-2

Let’s continue the SASL configuration.

$ sudo mkdir -p /var/spool/postfix/var/run/saslauthd
$ sudo rm -rf /var/run/saslauthd

Create the file /etc/default/saslauthd.

$ sudo touch /etc/default/saslauthd

Use your favorite editor to edit the new file so that it contains the lines which follow. Just to be clear, the final line to add begins with “MECHANISMS=“.

# This needs to be uncommented before saslauthd will be run
# automatically
START=yes

PWDIR="/var/spool/postfix/var/run/saslauthd"
PARAMS="-m ${PWDIR}"
PIDFILE="${PWDIR}/saslauthd.pid"

# You must specify the authentication mechanisms you wish to use.
# This defaults to "pam" for PAM support, but may also include
# "shadow" or "sasldb", like this:
# MECHANISMS="pam shadow"

MECHANISMS="pam"

Save the file.

Next, update the dpkg state of /var/spool/portfix/var/run/saslauthd. The saslauthd init script uses this setting to create the missing directory with the appropriate permissions and ownership. As earlier, sudo yourself to root and logout from root when finished. Be careful, that’s another rather long command that may have wrapped on your display.

$ sudo -i
# dpkg-statoverride --force --update --add root sasl 755 /var/spool/postfix/var/run/saslauthd

Then press ctrl-D to logout the root account.

Test using telnet to connect to the running Postfix mail server and see if SMTP-AUTH and TLS are working properly.

$ telnet foo.wan 25

After you have established the connection to the postfix mail server, type this (substituting your server for mine, of course):

ehlo foo.wan

If you see the following lines (among others) then everything is working perfectly.

250-STARTTLS
250-AUTH LOGIN PLAIN
250-AUTH=LOGIN PLAIN
250 8BITMIME

Close the connection and exit telnet with this command.

quit

We’re almost there, promise.

Setting External SMTP Server Credentials

Remember, we set out to use an external Internet-connected SMTP server as a mail relay and this is how that is set up. I mentioned at the beginning of the article that I had set up a dedicated account on one of my domains. You might use one on your ISP. I would not, however, use your usual email account.

You’ll need to manually edit the /etc/postfix/main.cf file to add these lines:

smtp_sasl_auth_enable = yes
smtp_sasl_security_options = noanonymous
smtp_sasl_password_maps = hash:/etc/postfix/saslpasswd
smtp_always_send_ehlo = yes
relayhost = [mail.example.com]:1212

Of course, you’ll modify the relayhost = line to specify your external SMTP server. If you don’t need a port number then simply leave off the colon and port number following the closing bracket. I included the port number as a syntax example in case you needed to use one.

Did you notice the hash file mentioned in the lines you just added to/etc/postfix/main.cf? It holds the SMPT server logon credentials, and it’s time to create it.

$ sudo touch /etc/postfix/saslpasswd

Use your favorite editor to edit the file, adding the credentials with a line like this:

mail.example.com username@example.com:yourpassword

The components of the line you’re putting in the new file should be obvious.

(Before you cry foul… Yes, I’m well aware of the risk of storing credentials in the clear. It’s a manageable risk to me in this case for the following reasons. The physical machine is under my personal physical control. The credentials are dedicated to this single purpose. If the server becomes compromised I can disable the credentials from anywhere in the world I can obtain an Internet connection. If I’m dead and can’t do that, well, I guess it’s SEP and my incremental contribution to the SPAM of the world will torment my soul until the end of time. Your situation may be different and I leave it to you to secure the credentials.)

Anyway, before postfix can use that horribly insecure file it needs to be hashed by postmap:

$ sudo postmap /etc/postfix/saslpasswd

With that done, restart postfix.

$ sudo /etc/init.d/postfix restart

Applications that know how will now be able to generate mail but it’ll be convenient to be able to do it from the command line. Besides making testing of this configuration easier you’ll then be able to have your own scripts send messages with ease. For that you’ll need just one more package.

Installing the mailutils Package

Simple. Install the mailutils package.

$ sudo apt-get install mailutils

That’s it!

Try test sending some email from the command line. Substitute the address at which you usually receive mail for my example youraddress@yourserver.com.

$ echo "body: outbound email test" | mail -s "Test Subject" youraddress@yourserver.com

Check your inbox.

Wrapping Up

Well, that wasn’t so bad.

VirtualBox on the 64-bit Ubuntu Server 10.10

Posted on by
Reply

I recently set out to upgrade a virtual host server from VMware Server to Oracle’s VirtualBox. The upgrade was a huge success. This is one of several articles where I talk about various aspects of that upgrade, hopefully helping others along the way. You might want to go back and read the introductory article Virtualization Revisited.

Installing Ubuntu Server 10.10 is very fast and straightforward – maybe 10 minutes tops. There’s no shortage of coverage of the install procedure so I won’t bother with it again.

But in case you’re not familiar, I’ll mention that the Ubuntu installer will offer to configure the server with a selection of packages right off the bat. Like many others, I prefer to do those configurations myself in order to tailor the instance exactly to my needs. I make an exception with Open SSH so I that can reach the server from the comfort of my desk by the time it’s booted itself for the first time.

So let’s assume you’ve just finished the IPL, popped the install media, booted for the first time and logged in. The very first thing to do is catch up on any pending updates.

$ sudo apt-get update
$ sudo apt-get upgrade

For the sake of completeness, if anything is shown as kept back you should probably do a distribution upgrade followed by a reboot. If not, skip ahead.

$ sudo apt-get dist-upgrade
$ sudo shutdown -r now

Next I install Lugaru’s epsilon editor, a very capable emacs-like editor that I run on all my boxes. Believe me: there’s great value in having one editor that behaves in exactly the same way no matter what keyboard’s under your fingers! I’ve been a Lugaru customer since the 80s and I’m pleased to recommend their rock-solid product. Go test fly their unrestricted trial-ware. Anyway, the epsilon installation needs to build a few things and installing this bit first allows that (as well as other routine software builds that might be needed in the future) to simply happen.

$ sudo apt-get install build-essential

To The Business At Hand: Installing VirtualBox

Download the key and register the repository for VirtualBox. The key has changed recently, so what you see here might be different from other articles.

$ wget -q http://download.virtualbox.org/virtualbox/debian/oracle_vbox.asc -O- | sudo apt-key add -

The key fingerprint is

7B0F AB3A 13B9 0743 5925 D9C9 5442 2A4B 98AB 5139
Oracle Corporation (VirtualBox archive signing key) info@virtualbox.org

Edit the file /etc/apt/sources.list to add the following lines, which simply adds the appropriate repository.

# VirtualBox 3.2.10 VirtualBox for Ubuntu 10.10 Maverick Meerkat
deb http://download.virtualbox.org/virtualbox/debian maverick non-free

Make your system aware of the newly added repository.

$ sudo apt-get update
$ sudo apt-get upgrade

Now you’re ready for the actual VirtualBox install.

$ sudo apt-get install virtualbox-3.2

Finally, add any users that will need to run VirtualBox to the vboxusers group.

Don’t forget the -a flag in the command! This is especially important if you’re manipulating your administrator account. (The flag indicates that the group should be added to the the account, rather than replacing any/all existing groups.)

$ sudo usermod -a -G vboxusers <username>

And that’s all there is to it!

[ed. Appended later...]

There have been a couple of comments in email about networking setup. “You must not be making your VMs visible to your LAN. There’s nothing mentioned about bridge adapters…”

In fact I am using bridged adapters in my VMs! Last time I looked at VirtualBox it was quite the pain to set up that way. When I came to that part I just gave it a WTF and tried to simply bridge eth0. It works just fine!

Thanks for asking.

Virtualization Revisited

Posted on by
Reply

I’ve been virtualizing machines the home network for many years. The benefits are simply huge (but relax – I’ll not go into them in detail here). Suffice it to say that it beats the snot out of stack of old PCs with their attendant noise and energy consumption.

The server I built on a shoestring one August afternoon many years ago has (ahem) served us well. A mile-high overview of the hardware includes an NVIDEA motherboard from BFG, several GB of commodity RAM, a SATA RAID card from Silicon Image driving a handful of 3.5-inch SATA drives, and an IDE boot drive. The mini-tower case – told you I cheaped out – is somewhat dense inside so there are extra fans to keep the heat in check. The host OS has been Windows 2000 Server Service Pack 4.

Yeah, yeah, I know. It’s a 32-bit OS on 64-bit hardware. A nice chunk of RAM is ‘lost’ to insufficient address space right off the bat. I figured to upgrade the OS one day but never quite got around to it. The virtualization software is VMware Server, which I’ve been using since the beginning. Their current version is 2.0.0 Build 116503 (wow, 2008, when dinosaurs roamed the Earth). The guest OSs are a mix of Linux and Windows servers handling core dedicated roles as well as a changing mix of experimental/test/research stuff: DOS, Windows 3.1, Chrome OS, OS/2 Warp (OMG what a hack that was!), a couple of OTS appliances, more. What can I say? I’ve got an interest in history. Besides, the look on my kid’s face when he sees an ancient OS actually running (as opposed to just static screen shots on some Web page) is worth it.

Anyway, there are lots of problems with this setup. VMware Server, their free product, is getting long in the tooth. The Web-based interface doesn’t work with the Chrome browser; it’s one of the few things that continues to force me to use IE. Sometimes the service side of the interface goes MIA altogether. The 32-bit Win2K is finally hopelessly out of date, absolutely no more updates. The list goes on and on.

So every now and again I look around for alternatives. The last serious contender was VMware’s ESXi. The idea of a supported bare-metal virtualization platform sure sounded appealing! I spent a day or two experimenting but ended up dismissing it. Getting it to run on the (albeit weak) hardware proved do-able but not without difficulties. In the end it just seemed too fragile for the long-term. I chalked it up to more trouble than it was worth, restored the old setup and got on with life.

The October 2010 issue of Communications of the ACM carried an interesting article, Difference Engine: Harnessing Memory Redundancy in Virtual Machines. Excellent article! A side effect of reading it led me to think again about the clunky mess humming away in the basement. And it was at roughly that time when another interesting article came through the news flow, How do I add a second drive to a Windows XP virtual machine running in VirtualBox?

Hmmm, VirtualBox. I had looked at VirtualBox a long time ago. I grabbed a current release and installed it on my desktop. Wow, it’s apparently matured a great deal since I last paid attention! I found it intuitive and fast to not only create and use new guests but also to simply import and run my existing VMs. (Well, okay, so there were a few gotchas, but no showstoppers.) Yes, this could be a contender for the basement server!

I pulled out an old laptop for some preliminary testing. I loaded it up with Ubuntu Server 10.10, installed VirtualBox and parked it in the basement. The goal? Well, VirtualBox is very easy to control through its GUI but I’d need to learn to run it entirely via command line and build my confidence for a smooth migration. I just  knew I’d run into problems along the way – nothing’s ever as easy as it looks at first glance – and I wanted to be able to anticipate and solve most of them in advance.

As usual, the ‘net came through as a truly incredible learning resource and I made copious use of it along the way. But every situation is different. By documenting my work in a series of articles, well, maybe it’ll help some wayward soul have an easier time of it.

Language Analysis, Anyone?

Posted on by
1

Pam‘s not much of a gamer but she plays The Sims. Has for years. Started with the first one, now they’re up to The Sims3. Quite a piece of software that is!

If you’ve played (or watched it played) you know that it’s a chatty game. That is, those simulated entities never shut up. Some of the sounds are universal. Babies crying, sounds of disgust (“Ugh!”) and so on. But conversationally they seem to have a language all their own.

I was wondering about that. First, does what they say have any consistency? By that I mean, say, when one of ‘em is hungry and mentions it, do they always say “oot grickle mem sitto zerk!” (or whatever that incomprehensible jabber is)? I don’t play, but I asked Pam and she said she thinks they might – but admitted she never paid attention.

By extension, if they do ‘speak’ with consistency then has anyone out there worked out the grammar? Is there anyone on the planet that can speak Sim?

Why not? There are people that can speak (and understand) Klingon. The ‘net delivers example after example of people that clearly have an abundance of free time. So why not?

Boosting SSD Performance

Posted on by
Reply

I’ve done some traveling this summer and the netbook I wrote about some time back has proved to be a worthy companion. The portability and battery life have more than offset the lower performance and cramped screen real estate. And the HP Mini 1000 has proven to be as reliable as a brick!

When I configured the box I chose the SSD over traditional hard drive. HDs tend not to last very long when transported via Milwaukee Vibrators. Sure, SSDs are considerably more expensive and offer less capacity, but I was looking for reliability and it’s certainly delivered that. Read speeds are fantastic, making for fast boot times even on the slow Atom processor. But small writes – the kind that Windows is famous for doing constantly – really suck.

I wanted to mention FlashFire, an SSD accelerator. According to their site, it’s “especially useful for the system using low-end SSDs.” It works. I haven’t bothered to upgrade the slow stock SSD mainly because FlashFire makes it tolerable.

Before you ask, yes, additional buffering can leave you with an increased risk of data loss if a crash occurs before the flush is complete. But the dirty little secret is that the higher-performance SSDs already use on-board DRAM buffers to boost performance, so is it really all that much different? I guess it depends on your needs. For me, the tradeoff – performance for a little more risk – is worth it.

If you’re grumbling and second-guessing your SSD decision, go give FlashFire a try.

TweetDeck and Blink182

Posted on by
1

I dunno, maybe it’s me. I use TweetDeck desktop client for Twitter. A while back I took one of their updates and blam! the colors went all butt-ugly and the sound went south. I’m not a Blink-182 fan, and it wasn’t a welcome change. I sort of dealt with it, and figured one day I’d bother to seek out another desktop client.

Well, today I accidentally found that the folks that make TweetDeck have realized that they made a mistake and have taken the high road. Check it out.

It’s fixed now and I’m happy again.

Now, if someone were to make a Frank Zappa theme…

Some Favorite Windows XP Registry Adjustments

Posted on by
4

Since I’ve been asked, here are a few of the registry adjustments I make soon after kickstarting an XP system. By no means is this an exhaustive list. No, it’s just the stuff that I consider a minimal start for all systems.

WARNING – Don’t come crying to me if you hose your system beyond belief, because for the uninitiated messing with the Windows registry directly is somewhat akin to performing open-brain surgery. In fact, I’m not going to tell you how to perform edits on the thing, back it up in whole or part or anything like that. You should already know how to do those things. If you don’t, well, please move along, nothing to see here.

With that out of the way, I’ll state what should be obvious. The registry keys mentioned below are each one line. Sometimes embedded spaces will cause wrapping that shouldn’t actually be.

The default responsiveness of the Start menu is designed for effect, not utility. Adjust it to your liking by adjusting the value here:

HKEY_CURRENT_USER\Control Panel\Desktop\MenuShowDelay

This has a default decimal value of 400. 100 usually does it for me.  The ever-so-popular TeweakUI utility adjusts this, too, but it’s easy to just do it this way.

If you’ve got enough memory in your system you can pull the Windows kernel into RAM. Absolutely don’t do this if you’ve got less than, oh, 256 MB.  But who doesn’t have 2 GB or more these days?

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager\Memory Management\DisablePagingExecutive

Choose one of these values:
1 = disable paging and run kernel from RAM
0 = normal, paged operation

It should be obvious that you want to set it to 1. You’ll need to reboot to make it take effect.

Did you know that NTFS maintains standard 8.3 file names that are compatible with DOS conventions? Those are the ugly looking all-caps things with the tildas and such that you may have seen in a file list every now and again. Creating and maintaining them is an overhead you can live without if you never have a need for this compatibility. Nice that you can easily disable it and keep your MFT a little less cluttered at the same time.

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\FileSystem\NtfsDisable8dot3NameCreation

0 = enabled
1 = disabled

Set to 1 to gain some file system performance, at the expense of compatibility with that older file system you probably forgot about long ago. You’ll need to reboot to make it take effect.

Oh, and before you ask: no, I’m not sure whether it cleans up existing 8.3 junk or not. I never bothered to check, but I’d suspect not.

Windows XP helps speed its bootup with a prefetch cache, located by default at C:\Windows\Prefetch. Some folks say that every now and again you should delete the contents of that directory, and the system will rebuild it cleanly. I personally wouldn’t bother with that, just let Windows deal with it. But you can control what gets prefetched with this adjustment.

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager\Memory Management\PrefetchParameters\EnablePrefetcher

0 = disable prefetching
1 = prefetch application launch files
2 = prefetch boot files
3 = prefetch as much as possible

Setting this to 3, of course, is a good idea.

The Disk Cleanup utility doesn’t actually clean up all of your temp files as you might be led to believe. Instead, it checks the last access of these files and if it’s 7 days or less it keeps ‘em around. Fortunately you can fix this.

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Explorer\VolumeCaches\Temporary Files\LastAccess

# = number of days of retention

Personally I like 0 days. One good reason is that it’s nice to have the slate as clean as possible when defragmenting. (But if you’ve got an SSD you might want to leave this one be, as small writes exact a serious performance hit.)

Add a Copy To command to Explorer’s context-sensitive menu, where it’s always ready for use.

Just add the following key:

HKEY_CLASSES_ROOT\AllFilesystemObjects\shellex\ContextMenuHandlers\Copy To

with a default value of
{C2FBB630-2971-11D1-A18C-00C04FD75D13}

And, while you’re at it, add a Move To command as well. Add this key:

HKEY_CLASSES_ROOT\AllFilesystemObjects\shellex\ContextMenuHandlers\Move To

with a default value of
{C2FBB631-2971-11D1-A18C-00C04FD75D13}

Of course, neither of these do anything for system performance but may help your performance.

Standards and Documentation

Posted on by
1

 

[This entry is lifted verbatim from a message I recently wrote in email to a good friend. We were idly discussing a bit of documentation that one of his technical writers had produced, when he commented that he created his own standards: whatever he said, so it would go. He concluded, "It's good to be the king, sort of..."]

I was lapsing into the way things used to be. Once upon a time there were Standards for everything.

Here’s a funny story. There’s no real proprietary stuff here, but it sheds a teeny tiny bit of light on the seedy underbelly of a company that would probably prefer otherwise.

Back in the 80s and before, there was a Standards Department. A handful of folks: a few writers, a few managers, a room of shelves with binders. (This was, of course, pre-LAN, pre-email, pre-all-the-stuff-we-take-for-granted-today. They walked floppies to a PC that was connected to an IBM line-printer. This was modern; not much earlier they used typewriters. The IBM ball-headed devices – were they called Quietwriters?  Selectrics - were still around.)

I hear you saying, “roomful of shelves with binders? Golly, what could they be documenting?”

Back then, every system, every subsystem, every sub-subsystem, every database, every data feed, every EVERYTHING was custom-built for a specific purpose – be it another system, a customer, whatever. This was before all the wonderful acronym-laden standards for such stuff we have today. (“I love standards – there are so many to choose from!”)

Anyway, time passes and in comes LANs and email and all kinds of magic and, one day, they went and dissolved the Standards Department. Figured that the Programmers could write their own documentation. Out went the writers, one by one. Then the managers. Their equipment was collected and taken away and their space was re-allocated. But not before I scoured their PCs for their documentation files. Thousands and thousands of Word docs. Stashed ‘em away in a big zipfile, I did.

Then there was the room full of shelves of binders. A girl I knew, a minor manager, was given the mandate to keep the lights on.

So the years passed. Major systems were rearchitected to common standards. New products were created. The outsourcing wave washed upon the tech shores. And lots of old talent – along with the knowledge of how the proprietary systems worked – was shown the door.

Along came Y2K, at first just a glimmer on the horizon. With the massive technical audit that was undertaken to prepare for that event came the realization that quite a bit of the shiny, new, “self-documented” code was critically dependent upon… wait for it… bits of old legacy stuff that nobody knew anything about anymore.

“Wait!” someone said, “We’ll call the Standards Department! All this stuff is documented!”

Uh oh.

It took a while, but eventually it was realized that the Standards Department had been decimated the better part of two decades earlier. Some hand-wringing later they discovered the roomful of shelves of binders. It had been dutifully passed along from hand to hand through several reorganizations, relocated over 2-3 facilities moves, but there they were. Unmaintained. Disorganized. Dusty. Thick, blue, three-ring binders, labeled with crusty, cryptic strings of numbers and letters – if you were lucky. Some had fallen off with age. But descend upon the room they did, borrowing one volume or another as the analysis plodded onward.

I remembered the original room, the old Standards Department, and when I heard about this I smiled. But when I heard that as often as not the borrowed volumes weren’t being returned, my smile turned into a frown. I grabbed control of the room, had it locked, began to mediate access. Soon I was doing a brisk side business as a librarian. I blew the dust off the forgotten zipfile and got the content onto the network. After all, it’s way easier to content-search a tree of files than to traipse over to some other building an spend hours with those dusty old binders. Or sign your life away to the shaved-head dweeb that made sure you brought ‘em back. Trouble is, the files and the binders ain’t exactly one and the same all the time.

And then, there’s the stuff that no one, try as they might, could find documented ANYWHERE. Several thousands of those entities were scattered across the organization. Little black boxes, you can see what goes in and comes out, but haven’t got an inkling of what goes on inside. Except when one little black box talks directly to or from another little black box, hmmm, then you don’t really know much about the interfaces either. Quite troubling.

Y2K came and went – rather uneventfully, actually. The world didn’t end. The systems actually came out the other side better than they went in. Life went on. Interest in the room and the files waned, but didn’t go away. As it turns out, Programmers, especially contractors, especially hourly contractors with lots of churn, aren’t exactly the best when it comes to documenting their work. And “self-documenting code” really isn’t, unless the reader is quite technical. The legacy stuff, well, the stuff that’s actually documented, turns out to be the best documented stuff there is. Created by people whose job it was to make it so.

Now here’s the punchline. To this very day, if you dig deep enough, through the shiny, new Web-enabled, SOAPed and serviced layers, you could very well discover dependencies upon some bit of legacy code or another that *nobody* understands, code for which there’s *no* source code, *no* documentation…

This is a good time to end the story, as we sit and sip our morning coffee, pondering the sinking feeling in the pit of the stomach of some poor sod somewhere whose unfortunate lot puts them near one of those bits of code.

Fighting The Good Fight

Posted on by
Reply

 

The amount of spam I’ve been receiving on this blog had been skyrocketing lately. It reached the point that it was pretty much an everyday chore to clear it out. So, like many before me I decided to activate the Akismet (version 2.2.1) plug-in.

All was well for a few days. But then, out of nowhere, Akismet began calling my attention to an unbelievable amount of trackback spam. By ‘unbelievable’ I mean several a minute, sometimes. Hundreds and hundreds overnight.

Now, that shouldn’t be a problem because they’ll go away on their own after a period of time. But what about legitimate stuff? There could be some of that, and it’s important to flag it so Akismet ‘learns’.  Um, that’s what they say, anyway. The trouble, of course, is that the longer the list of stuff to look over becomes, the harder it is to identify the good stuff.

This morning I logged on to see 17 l-o-n-g pages of it. Something would have to be done!

Here’s what a typical entry on the Akismet Caught Spam page looks like.

All instances share the IP address of 82.233.30.32 which is linked to a whois search. If I point my browser at the IP directly I see a typical Apache test page – the server the offending server is powered by CentOS. A reverse-DNS doesn’t give any more insight – no other host names. Google doesn’t have it cached, either. The IP is probably spoofed…

The text of the spam changes a bit, as does the host name. When I point my browser at the host name, though, there’s some kind of content for just the briefest instant, but then it quickly changes to a typical blog has been removed page. In fact, every one I’ve looked at is exactly like this.

Whaddya know, onlinecasino21.blogspot.com doesn’t resolve to the IP address I mentioned earlier, either. What a surprise, right?

Anyway, it would be nice if Akismet allowed you to filter the spam and apply a delete all to the result. But it doesn’t, so we’ll have to take more drastic measures.

Turning off trackbacks and pingbacks (same setting) would probably work but I’d rather not do that. Blacklisting the address in WordPress doesn’t work, Akismet still gets it first. Here’s what I did. In my .htaccess file I added these sections.

And that seems to have applied the brakes. I haven’t seen another instance of this spam for several hours.

Another thing that just might be worth mentioning. I run several blogs and when I was activating Akismet to mine I activated it on the others as well. But this – my personal blog – is the only one that’s been troubled by this onslaught of trackback spam. I don’t know who I pissed off out there, but somebody – or something – has latched on and it ain’t letting go.

Virtuality

Posted on by
Reply

Well, VMware Server 2′s been out long enough without panic-updates so I finally got around to upgrading one of the servers.

There were only five VMs on the target box; the backups – about 250GB worth – went quick enough, disk-to-disk. The VMware software on the Win2K host also went rather uneventfully. Then the fun began.

There’s no standalone management console now, all that stuff is done through a Web interface. I like the Web as much as the next guy, but let’s face it: it’s slower. I haven’t had any trouble with it – yet – but I’m waiting. Next, the remote consoles to the VMs are implemented as a browser plug-in. Fair enough, but try as I might I’ve been unable to get the plug-in to be called by Chrome. I thought I’d have to use IE (it installs fine on IE7) but then I found that one can generate a shortcut that calls the plug-in exe file (my laptop runs XP). The end result is that I can manage the host with Chrome and call VM consoles up as needed. Well, the Windows VMs, anyway. The Linux VMs are fine, as usual with SSH.

Then there’s the VM updates themselves. It’s a one-way process (another reason to have good backups!) and you get a reasonable warning before you proceed. Of course, when the VM’s OS wakes up quite a bit of the virtualized hardware has changed. That means driver changes and such, it’s as though you changed motherboards or something equally traumatic. In my case it all went okay, with one exception. A Windows Server VM would no longer start SQL Server 2000 for lack of a DLL: msvcp71.dll. As it turns out I had one handy – quite accidentally, I assure you – so I copied it to the VM’s WINNT directly and all was well again.

I generally use the VM Tools, too, so those were next. The updates were intuitive, but different. From the Server management interface, the necessary files are placed on the VM’s CD-R drive. Then, from the VM, you install from there. Now, there’s been one Ubuntu VM that I’ve never been able to install Tools on for some reason. Never could figure out why and it wasn’t important enough to pursue. This time I simply mounted the drive and everything went flawlessly. Go figure.

All the slogging complete it was time for some testing. I’m pleased to report that every VM is showing solid signs of performance increases across the board! Memory management seems significantly improved, as does virtual disk performance. It’s too early to be saying anything about reliability, of course, and I have yet to experiment with other new features. I may even eventually get used to the Web management interface.

So there you have it. Not bad for a couple of hours of work. VMware Server 2.0 is a free download. If you’ve got a spare box hanging around and always wanted to play with virtualization, go give it a try.

Laptops and Hard Drives

Posted on by
Reply

My wife’s laptop was getting full. NTFS, as you probably already know, begins to suffer performance-wise when it crosses the half-full line. And the default MFT size is kind of small to begin with. Presently that all-important area was about 98% consumed and the drive itself had only 20% or so free space. Her last install of a Sims2 expansion pack brought another round of complaints.

Easy enough to remedy. Head out to Best Buy for a replacement drive. But how to get the new drive installed and set up as pain-free as possible? Usually it’s a fresh IPL, but I was looking for the easy way out.

I have this neat device from CoolMax. The CD-350-COMBO is a multi-headed cable that plugs into a raw IDE or SATA drive and presents to your system as a USB device. When your laptop is your workbench this device is worth its weight in gold. Soon the new drive was partitioned, formatted, and tested. (For good measure, I allocated a much larger MFT as well.)

With that problem solved I turned to the task of cloning the existing drive. I recently read of something called XXCLONE, which promised a file-by-file copy (including all the locked stuff) from a running Windows system, with the ability to make the destination bootable. This would be a good time to try that out.

The install to the wife’s laptop was easy enough: unzip and copy a file. I used the CoolMax adapter to cable up the new drive, the destination for the copy. I set XXCLONE to task and went away. The copy would take a while. When I returned it was finished. I made the new drive bootable with a couple of clicks, uncabled and shut everything down. It took a few more minutes to physically swap the old drive for the new one.

The first boot took a little longer than usual. Windows was a little confused, I guess, because the drive change triggered the New Hardware Wizard. But soon things settled down. Between these two tools, a usually-tedious job was turned simple!

There’s one other thing I should mention. The XXCLONE documentation claim that because it makes a file-by-file copy, it defragments the destination drive automatically. I run Diskeeper on all of our machines, and it reported the drive as heavily fragmented. I needed to run the boot-time defragmentation job before the new drive delivered its expected performance.

Additional stuff, 17 December 2008: There were a couple of nagging issues following the drive cloning. I’m not sure if it’s XXCLONE or if it’s integral to the cloning process itself, but some applications installed with the MS Installer were no longer accessible through Add/Remove Programs. Instead there would appear a dialogue:

“The patch package could not be opened. Verify that the patch package exists and that you can access it, or contact application vendor to verify that this is a valid Windows Installer patch package.”

The solution, while a bit of a pain, is to obtain and install the Windows Install Clean Up utility from Microsoft. Run the utility and select the errant application from the list, then clean it up – which amounts to removing it from the installer’s database. Finally, re-install the application.

In my case it was Office 2003, which called for finding the license number and install media as well as a few rounds of patches and service packs. There were a few other applications as well, but that was the most substantial.