Hey, are any of my NYC-area geek friends interested in a 2-week contract sysadmin gig? The client is a friend and former boss of mine, and they need someone ASAP. Drop me a line and I’ll put you in touch.
All articles, tagged with “the biz never sleeps”
If you ever get the chance to acquire founders’ stock in a company, take it.
“chance” implicitly including hopefully obvious baseline requirements like “can still pay for food and rent”
The following is 100% geek-speak, and I’m posting here because it’s the solution to a problem that bedeviled me for several days, and as far as I can tell the method is not to be found anywhere via the expected search terms. Hopefully this post will fix that.
Late last year, for the first time in about a decade, I bought a completely new-in-the-box computer: a Mac Pro. (A purchase that was, thankfully, paid for by people who were not me — an amusing story for another time.) It is, hands-down, the best computer I have ever owned. But there was, of course, one small crazy-making problem:
When connected to my Dell LCD monitor (a 2001fp: still quite the respectable little beast, and connected via DVI, so you’d think that the computer and monitor would be able to figure this stuff out), the first-stage boot screen (dark grey apple, light grey background, spinner) did not display. The screen stayed entirely black until it got to the second-stage screen (“Welcome to OSX”, blue background, blue progress meter) and the login window. Sometimes, if the moon was in the right phase, the Dell would display a “can not display this mode” error, but usually it would just stay black.
Now this would have merely been an aesthetic oddity, except that the video mode that the monitor wouldn’t display is the same video mode used for several actually important things, ie:
1. verbose boot: all black
2. single-user boot: black on black
3. Boot Camp selection screen: none more black
The last one, as you can imagine, is a bit of a downer, since the system will happily pause endlessly mid-boot waiting for you to pick a boot volume, which is hard to do when you can’t see anything. (I got pretty good at doing this by keyboard and intuition, but it’s not a skill I’m happy to have picked up.)
Digging around on google suggested that the problem was endemic on older Dell LCD monitors, and more likely to happen to Mac Pros with the ATI Radeon graphics card installed, but I wouldn’t claim either as a 100% certainty.
Well, it took me a while, but I finally sussed it out. The fix for this is documented precisely nowhere on Apple’s site, and the only hints I found of it were in the archives of a bunch of sites dedicated to hacking OSX to run on non-Apple hardware. Even better, not one but two things were wrong, and it took me a while to tease them both out.
First: the DVI cable that came with the Dell monitor, and which I was still using many years later, was a “single-link” DVI cable. This, apparently, is a no-no: the first stage boot screens require a dual-link cable to display on this monitor. Why? I have no idea.
Second, the video card and the monitor were failing to handshake properly. It turns out that you can manually specify the resolution and refresh mode of the first-stage boot process in the /Library/Preferences/SystemConfiguration/com.apple.Boot.plist file, like so:
<?xml version=”1.0” encoding=”UTF-8”?>Pick a resolution and refresh rate that you’re confident your monitor can support, save the file (you’ll have to be root to edit it, of course) and reboot. Et voila, the boot screen is back.
<!DOCTYPE plist PUBLIC “-//Apple Computer//DTD PLIST 1.0//EN” “http://www.apple.com/DTDs/PropertyList-1.0.dtd”>
<plist version=”1.0”>
<dict>
<key>Kernel</key>
<string>mach_kernel</string>
<key>Kernel Flags</key>
<string></string>
<key>Boot Graphics</key>
<string>Yes</string>
<key>Graphics Mode</key>
<string>1280x1024x32@60</string>
</dict>
</plist>
Obvious warning: playing with system boot parameters is a great way to turn your computer into a large, expensive, electricity-sucking brick. Take a full backup before you attempt any such surgery, and please don’t email me for support.
First, a slightly belated update: the spam flood to my server has been stopped in its tracks. Email and web services are back to normal, for localized values of ‘normal.’
Now, a small statistic, followed by a rant. This is high geekery; everyone not interested in hearing me gripe about mail servers should probably skip it.
Since this Thursday afternoon (when I finalized the new configuration), there have been 23,540 attempt to deliver mail to my mail server that were not pre-blocked by the spamhaus.org or sorbs.net antispam blacklists. Of those attempts, 19,166 of them were attempts to deliver mail to addresses which do not exist. Either they were “dictionary spam” (in which the spammer goes through a list of common first and last names, and attempts to deliver spam to every possible name@domain.com), or “bounceback spam”, in which the spammer invents a fake address at someone else’s domain and forges that as the sender of the spam, so that any complaints or bounces go to the forged address instead of the spammer.
For those of you without a pocket calculator in the audience, that means that 81% of the emails sent to my mail server were to addresses that never existed.
This would merely be cause for hilarity, except that a design flaw in qmail, for years my mail server of choice, turned it into a complete debacle.
You see, the stock distribution of qmail does not validate the user portion of an email address until after the mail is accepted. In plainer English, that means that if you send mail to bob@blank.org, and I’m running qmail on blank.org’s mail server, qmail will happily accept that mail even if there is no such user as “bob”. Even if that mail is spam. Even if that mail contains a 300k image file. Even if that mail is spam and contains a 300k image file. And then, once qmail finally realizes that there is no Bob, qmail generates a new piece of email, a bounce message destined for the original sender, letting them know that the message wasn’t delivered, so not only have we spent time, disk and network bandwidth accepting a message that we never wanted in the first place, but we’re then going to spend MORE resources sending out an alert, quite probably to a sender address that was forged by a spammer in the first place.
Multiply that by thousands to millions of bogus messages a day and you have a problem. Add on top the CPU and disk utilization required by any minimally-responsible set of spam- and virus-filtering tools (spamassassin, crm114, clamav, etc) and you have a disaster. I run a very small mail server — a handful of domains, less than 20 users, less than 10 mailing lists — and the increased system and network load caused by accepting all of that bogus mail essentially took me off the air for a week, and came perilously close to blowing my ISP’s network quota and thus costing me quite a bit of money.
This was a defensible design decision in 1996, when you couldn’t sneeze without uncovering a new buffer overrun in sendmail, and massive email spam was a distant thundercloud on the horizon. Today? Not so much.
Luckily, this is a problem that has been solved a few times now. The qmail-validrcptto patch is probably the simplest way for most qmail administrators; if you’ve got some horribly abstruse setup going, it might be easier to use the qmail-spp plugin patch and write your own validator for the RCPT stage. But whatever you do, you should fix the problem: if you’re running a stock qmail or netqmail right now, you are acting as a spam amplifier whether you realize it or not.
Well, it’s finally happened. The sheer amount of spam that my mail server gets has reached the point where not only is the spam itself nearly overwhelming my network bandwidth, but all of the clever anti-spam crap I run on the server is crushing the CPU.
I’m working on a New Clever Trick to deal with this, but mail to and from my domains (blank.org, sneaker.net, etc) is going to be kinda crappily slow for the next few days at least, as will access to any of the websites hosted there.
Even better, Gmail has apparently decided that my server is a spam source, so all mail from my users to gmail accounts is getting tagged as spam. Luckily, I can personally harrasss the anti-spam team there, so that might get resolved sooner.
Remember: kneecapping spammers is the holiday gift that keeps on giving.
Some kind soul decided to dump SEVENTY-EIGHT THOUSAND spam messages destined for a domain I provide backup mail service for onto my server.
Mail to or from me, or any domain hosted on my server, may be a bit laggy for the next few hours as I dig throught he mess.
Remember: kneecapping spammers is a mitzvah.
Needed: one senior solaris sysadmin-type geek in the NYC area, looking for full-time or 6-12 month contract work. Must be a fast thinker, quick on your feet, with a strong knowledge of perl, bash, apache and the rest of the usual unixy toys.
If this sounds like you, or someone you know, drop me a line.
(My non-geeky readers can avert their eyes now. Trust me, you will not find this interesting.)
A lot of people use RT to track helpdesk requests, problem reports and other incident data at their jobs. An even larger number of people use or are forced to use Microsoft Active Directory as the central repository of username and password information at their jobs. As a result, probably the single most-asked question on the rt-users mailing list is “how do I unify logins between RT and ActiveDirectory?” Strangely, it’s a question that seems to lack an authoritative answer.
Until now. Ladies and gentlemen, I proudly present:
The holy grail: Single Signon RT with Active Directory
(Severe nerdery ahead; turn off now.)
John Gruber (no, not the arch-kook of late-1990s Usenet: that was John Grubor) has a great article up on his site talking about one of the great shibboleths of this industry.
In a nutshell, if you ask any geek, industry pundit, or even a random person on the street, “What did Apple do wrong in the 1980s?” they’ll pretty much all say the same thing, because it’s been repeated so many times that it’s become Received Wisdom:
“Apple should have licensed the Macintosh OS, or ported it to Intel hardware.”Everyone “knows” this: if Apple had just ported the MacOS over to the PC AT and/or licensed it to all comers, they would be Microsoft now.
Now, Gruber goes into a great number of the reasons why this is nowhere near so “obvious” as it seems, and he’s entirely right about them. (I won’t rehash his points here: go read it yourself.) But to my mind, he’s forgotten the most obvious case against this argument: other people did this, and it didn’t work.
GEM. GEOS. DesqView. DesqView/X. Remember any of these names? Unless you’re a pretty hardcore geek, the odds are that you don’t, and that’s fine, because there’s no reason you should: in greater or lesser degree, they all did the same thing (put a vaguely Mac-like interface onto a standard desktop PC circa 1986), and they were all, in the long run, total failures. Digital Research, Berkely Softwarks and Quarterdeck Software, their respective makers, are all historical footnotes now: long-dead or absorbed into larger firms.
Heck, for at least a few years there, even Microsoft couldn’t sell a graphical interface on a PC to save their lives. Remember Windows 1.0? Windows 2.0? Windows 386? If you’re lucky, you don’t. If you had to use any of them, I’m sorry.
Seriously: if “an integrated GUI desktop on a commodity PC hardware platform” were really all that you had to create in order to make yourself into the Richest Company Ever, then by all rights Digital Research Inc should be a world-straddling colossus right now, and Microsoft a piddling vendor of second-tier office productivity software. But that’s now how it worked out, and people looking for glib explanations for Apple’s shifting fortunes would do well to look elsewhere.
For the “benefit” of whatever person will succeed me at my current job, I’m assembling a list of questions that will be presented as a quiz to all applicants. Basically, the idea here is that if you really are a 5-to-7-year UNIX sysadmin, you should probably know a lot of this stuff off the top of your head.
- On a Sun Solaris system, how do you promote an NIS client to an NIS slave server?
- On Sun Solaris, how do you create a central mail relay smarthost without changing the sendmail configuration on any other machine?
- On Sun Solaris, how do you change the order of name resolution between DNS, NIS and the local hosts file?
- (Followup) How might you do this on a system that doesn’t support the Sun/SVR4 method?
- On Sun Solaris, how do you reset all network-related configuration information?
- How do you non-destructively reset the root password on a Sun server who’s root pw has been lost? Assume that you have Solaris installation media.
- On Red Hat Linux, how do you assign a static address to an interface which is currently assigned an address by DHCP?
I solicit from my geeky friends: got any other questions that you think should be asked? Think that any of these questions are horribly unfair?
N.B.: I’m looking for questions that can be answered, definitively, by a series of steps/commands. More open-ended stuff like “describe some key differences between FreeBSD and Solaris” is, to my mind, more suited to the actual in-person interview itself.
