Which directories were archived? Command-line
Posted: 27 July 2011 Filed under: productivity | Tags: command line, linux Leave a commentI archived a directory. It took two hours, then exited with a non-zero exit status (that means an error). Hmm — I was just testing something; I only cared if certain specific subdirectories were present in the archive. So I needed a way to look deep inside, quickly, and find those particular directories.
GNU tar will let you “test” an archive with -t, but I only wanted a list of the directories archived. Then I wanted that sorted. So…
$ nice tar -tjvf data.tar.bz2 | tr -s ' ' | cut -d' ' -f 6- | cut -d / -f -2 > tardirs.txt
$ uniq tardirs.txt > tardirs_uniq.txt
$ sort tardirs_uniq.txt > tardirs_uniq_sorted.txt
The -tjvf arguments to tar let you look inside, the “tr” command collapses adjacent spaces so that the first “cut” command will output only the sixth (file) field, and the second “cut” command will reduce a directory like “folder/folder/folder/fun.txt” to “folder/folder.” Then “uniq” will remove non-unique names.
Ignoring Ubuntu Upstart for System V compatibility
Posted: 21 July 2011 Filed under: system administration, ubuntu 2 CommentsIf you’d like to start an Ubuntu service using the System V-compatible runlevels, use “update-rc.d.” For example:
# update-rc.d ssh defaults
update-rc.d: warning: ssh stop runlevel arguments (0 1 6) do not match LSB Default-Stop values (none)
Adding system startup for /etc/init.d/ssh ...
/etc/rc0.d/K20ssh -> ../init.d/ssh
/etc/rc1.d/K20ssh -> ../init.d/ssh
/etc/rc6.d/K20ssh -> ../init.d/ssh
/etc/rc2.d/S20ssh -> ../init.d/ssh
/etc/rc3.d/S20ssh -> ../init.d/ssh
/etc/rc4.d/S20ssh -> ../init.d/ssh
/etc/rc5.d/S20ssh -> ../init.d/ssh
Alternating row backgrounds / conditional formatting in OpenOffice.org
Posted: 26 May 2011 Filed under: open source, productivity | Tags: openoffice.org 8 CommentsHave you put together an OpenOffice.org table in OpenOffice.org Calc, and wanted to display alternating backgrounds for easy reading? Well, there’s an easy way to do it.
Row backgrounds alternating white and gray 10%
The steps are:
- Create a style for every other row and call it “Even rows.” Make its background “Gray 10%,” or whatever other color and attributes you prefer.
- In your spreadsheet, select the range of rows you’d like to format.
- In the Format menu, click “Conditional Formatting…”.
- Choose “Formula is” in the first dropdown.
- For the condition itself, enter the formula
ISEVEN(ROW()). This formula tests if the current row in question is even or not. If it is, the condition is met and your style is applied. - For Cell Style, select the style you created back in Step 1.
- Click OK. There you go.
Your formula should look like this.
Google Datacenter Security
Posted: 26 April 2011 Filed under: security, system administration | Tags: security Leave a commentJust saw what basically amounts to a marketing video for Google’s Apps customers, discussing some of the features of their datacenters. They’ve put more thought than you’d expect for these facilities, specifically the “sustainability” aspect — which may involve a little bit of greenwashing, but I suppose it’s a start. To recap:
- Physical barriers and perimeter fencing
- 24/7 security (I bet they have free coffee!)
- Access control with badging and biometrics (such as retinal scans)
- Local law enforcement response
- Video monitoring and “video analytics” (motion and object detection, or video tracking)
- Data protection (encryption would have prevented HealthNet’s embarrassing incident, their second in two years)
- File fragmentation, replication and storage (including “sharding” to increase data security)
- Hard drive lifecycle management (how they destroy drives with a crusher and recycle the pieces)
- Fire detection and suppression
A drive crusher: 
SSH postponed authentication (publickey)
Posted: 25 January 2011 Filed under: security | Tags: security Leave a commentAn interesting syslog message!
sshd[xxx]: 'Postponed publickey for oracle from ::ffff:130.156.249.64 port 9264 ssh2'
As Tom explains,
ssh is just stepping through the client-configured authentication types (enabled in either ssh_config or ~/.ssh/config) and trying them in turn, but acknowledging that a publickey was presented, and postponing its use until after trying the first auth type.
Split a file across multiple CDs or DVDs on Linux
Posted: 13 January 2011 Filed under: productivity, ubuntu | Tags: command line Leave a commentOkay, so you very likely have the ‘split’ utility installed (it’s in the GNU coreutils package, so… very likely). If you want to burn a file to multiple media, but you don’t have kdar installed on your desktop… don’t worry about it. Just open a terminal and do:
$ split --bytes=600MB --numeric-suffixes filename.zip filename_part_
In my case, I have a 2.8GB file, but I only have 700MB CDs on hand for my burner. So this command will ensure that I get several 600 “megabyte” (1000 bytes * 1000) pieces, named “filename_part_00,” “filename_part_01,” and “filename_part_02,” et cetera.
User-friendly PKI: not yet
Posted: 12 November 2010 Filed under: security | Tags: gpg, pgp, pki, privacy, security 2 CommentsPublic-key infrastructure is great. When it works, you can be quite sure that you are communicating with whom you want to communicate, and you can even communicate securely if you wish. Alice and Clare can send each other mail, and Bob won’t be able to sneak in there and read it.
It’s supremely easy for Alice to create a private key, and publish her public key to a keyserver. “That’s it?” she thinks, with a leap of her heart. “Wow! It’s so easy!”
Let’s fast-forward a bit.
For a while, Alice and Clare have fun sending encrypted messages back and forth. They find a plug-in to send encrypted mail through their GMail accounts (FireGPG), and that’s useful for a while, although oops, the developer discontinues GMail support. But they can still use other clients, and they do so.
Things are hunky-dory for a while.
Then one day Alice upgrades her computer, and she forgets to migrate her private key over to the new box. She doesn’t have her passphrase at hand, of course, because she had her e-mail client save it for her; she had it on a sticky note so she only had to look it up once in a while.
“Oh, well,” thinks Alice. “I’ll just create a new public key.”
Oops. Is this user-friendly?
Now Alice has a PGP key sitting out there on a keyserver. She can’t use it because she doesn’t have the equivalent private key. She can’t revoke it because she didn’t think it was that important to create a revocation certificate at the time and save it. Someone told her to do that once. “Don’t save it in the cloud, but on a disk that would never lose its data and could never be compromised. Then lock it away in a safe in your basement. You did make sure your basement was a Faraday cage, didn’t you?”
PKI is such a useful, powerful tool. But… it’s still very easy to screw up and cause lasting damage, though. I guess it’s like a “Hole Hawg,” an apt comparison to other powerful technologies, as noted by Neal Stephenson.
There IS an option for Alice. She can create a new key, with a user ID like: “only use this key. That other one is bad,” and then get all her friends to sign the new key. There are other techniques, but that’s pretty much as close as she’s going to get.
Questions like this in security-related topics interest me: If it’s easier to use, will more people use it? And if it’s more widely used, will that increase its relevance?
Are distractible people more creative?
Posted: 12 October 2010 Filed under: productivity | Tags: creativity, focus, self-discipline Leave a commentAre Distractible People More Creative? Jonah Lehrer, Wired Science
He says it right: “We need to be ruthless about throwing out the useless stuff.”
How do great authors write? And great painters paint? They focus. Distraction might flit an idea in your face, but it doesn’t have the power to lead anyone to great production. Just more distraction and dissipation.
Discipline, however, plays a much greater role:
“I write when I am inspired. Fortunately inspiration strikes every morning at 9 a.m. sharp.”
– Somerset Maugham
Bash pipe fun
Posted: 16 September 2010 Filed under: productivity | Tags: bash, command line, linux 1 CommentHow about “recursively look at a log of hostnames used to request my site content. Sort them and ensure that only unique ip address and hostname combinations are counted. Find how many use my ‘.biz’ hostname to land on my site”:
find . -iname '*ecommerce-host_log*' | nice cat | nice xargs cut --delimiter=' ' -f 1,4 | nice sort | nice uniq | nice grep \.biz | nice wc -l
I wasn’t sure which commands would be most processor-intensive, so I used “nice” liberally.
Apache custom logging
Posted: 31 August 2010 Filed under: system administration | Tags: apache, bash, command line, linux, system administration, web analytics Leave a commentAren’t you interested in seeing what requests users, bots, or script kiddies make of your site, especially those things that client-side JavaScript-based analytics packages don’t tell you?
Under Apache, custom logging can give you lots of information you might not have seen otherwise. I’ll let the documentation for Apache’s mod_log_config say most of this, but as a quick preview, you could try defining a custom log format up near the top of your httpd.conf with
LogFormat "%a %t %{Host}i \"%r\"" hostlog
for example, then in all of your Directory containers, you could do
CustomLog logs/forest-monsen-site-host-log hostlog
Then, in my case, /var/log/httpd/forest-monsen-site-host-log would contain lines like
192.168.0.3 [31/Aug/2010:08:53:24 -0500] www.forestmonsen.com "GET /aggregator/sources/2 HTTP/1.0"
192.168.0.5 [31/Aug/2010:08:53:24 -0500] www.forestmonsen.org "GET /images/house.gif HTTP/1.1"
And I’d be able to tell which hostname was originally requested by the user — before any of my mod_rewrite rules got to it. Good stuff.
Emphatic Nonsense 