tinyapps.org / blog

Deduping files in Mac OS X #

While recovering data, Photorec terminated prematurely with a "segmentation fault" error. Data Rescue was run on the same drive, and while it completed successfully, I wanted to merge and dedupe the files recovered by both apps.

For deduping, Adrian Lopez's open source fdupes was used (sudo port install fdupes*). It first compares file sizes and MD5 signatures, and then performs a bit-for-bit comparison in case two different files have the same signature.

To process filenames that contain spaces, Geekpoet suggests setting the internal field separator (IFS) to $'\n'
$ IFS=$'\n'
$ for i in $(fdupes -f ./); do echo deleting: $i; rm $i; done
I used the single line option (-1) and xargs instead:
$ fdupes -rf1 ./ | xargs rm
The Big Mean Folder Machine then merged (and automatically renamed where necessary) the remaining 20,000+ files into a single directory. However, the Finder choked when opening the directory, so two new directories were created and files moved like so:
$ find ./ -size -128k -exec mv {} ../small/ \;
$ find ./ -size -1024k -exec mv {} ../medium/ \;
Since the remaining files were all over 1024k, the directory was simply renamed to "large".

* Use MacPorts, avoid DarwinPorts .com.

/mac | Dec 24, 2010

LAN instant messaging #

/windows | Dec 12, 2010

nweb: a tiny, safe Web server (static pages only) #

"Have you ever wanted to run a tiny, safe Web server without worrying about using a fully blown Web server that could cause security issues? Do you wonder how to write a program that accepts incoming messages with a network socket? Have you ever just wanted your own Web server to experiment and learn with?

Well, look no further -- nweb is what you need. This is a simple Web server that has only 200 lines of C source code. It runs as a regular user and can't run any server-side scripts or programs, so it can't open up any special privileges or security holes."
(via Mohan)

/nix | Dec 12, 2010

Kindle Terminal #

Luigi Rizzo has crafted a standalone Kindle terminal in C that allows you to run a shell from the Kindle's keyboard and screen. No need to rely on web-based terminals any more. (via Hacker News)

UPDATE: Installing Luigi Rizzo's Standalone Kindle Terminal

/eink | Dec 11, 2010

Recover Internet Explorer autocomplete form strings and more from offline systems #

Passcape Internet Explorer Password Recovery (PIEPR) recovers data cached by Internet Explorer, including: cached credentials, FTP passwords, AutoForm and AutoComplete fields, AutoComplete passwords, synchronization passwords, Identities, and Content Advisor passwords. Recovery can be run on live or offline (via ntuser.dat) systems. See also Recovering Internet Explorer passwords: theory and practice.

UPDATE: Firefox's autocomplete form strings can be found in formhistory.sqlite (which is located in the user's profile folder). Easily view and extract data with the free and open source SQLite Database Browser.

/windows | Dec 08, 2010

Over 2,000 years later, science catches up with the Stoics #

Radiation Rings Hint Universe Was Recycled Over and Over
"Most cosmologists trace the birth of the universe to the Big Bang 13.7 billion years ago. But a new analysis of the relic radiation generated by that explosive event suggests the universe got its start eons earlier and has cycled through myriad episodes of birth and death, with the Big Bang merely the most recent in a series of starting guns."
The Stoics by John M. Rist:
"Each of the early Stoics taught that the universe, after certain definite periods of time (whose duration is never specified), dissolves into fire and so 'purifies' itself ... We can see how this process would have been outlined by Zeno: the universe originally generated from creative fire (pyr technikon); after a period of time it resolved itself once again into pure fire, whence another universe was in turn generated from the pure creative fire. The cosmic process is therefore an infinite series of finite phases of alternate diakosmesis and ekpyrosis."

/misc | Nov 30, 2010

Editing or deleting file metadata in OS X #

The Finder's Get Info window does not allow changing most metadata, like the URL in "Where from" as shown here:

mdls can list a file's metadata attributes:
$ mdls file.flv
kMDItemContentCreationDate     = 2009-07-30 17:48:06 -1000
kMDItemContentModificationDate = 2009-07-30 17:48:06 -1000
kMDItemContentType             = "com.adobe.flash.video"
kMDItemWhereFroms              = (

but cannot edit them. For editing, we need xattr:

$ xattr -l file.flv
00000000  62 70 6C 69 73 74 30 30 A1 01 5F 11 01 7F 68 74  |bplist00.._...ht|
00000010  74 70 3A 2F 2F 73 31 73 64 6C 6F 64 30 35 33 2E  |tp://s1sdlod053.|
00000020  62 63 73 74 2E 63 64 6E 2E 73 31 73 2E 79 69 6D  |bcst.cdn.s1s.yim|
00000030  67 2E 63 6F 6D 2F 2F 73 31 73 6E 66 73 30 36 72  |g.com//s1snfs06r|
00000040  30 35 2F 30 30 35 2F 76 69 64 65 6F 73 65 61 72  |05/005/videosear|
00000050  63 68 2F 39 30 35 32 32 35 37 37 2E 66 6C 76 3F  |ch/90522577.flv?|
00000060  53 74 72 65 61 6D 49 44 3D 39 30 35 32 32 35 37  |StreamID=9052257|

$ xattr -w "com.apple.metadata:kMDItemWhereFroms"
  "http://topdocumentaryfilms.com/journey-edge-universe/" file.flv

-d can be substituted for -w to delete the data. More info on OS X metadata and extended attributes

/mac | Nov 30, 2010

Cracking OS X keychain files #

KeychainRecovery is a free tool to recover OS X keychain file passwords. (via What's My Pass?)

Especially useful after having reset a user's password via installer DVD or single user mode. The default keychain file (login.keychain) is normally located in ~/Library/Keychains.

It is not necessary to run the installer; simply extract KeychainRecovery.app from Archive.pax.gz found inside KeychainRecovery.pkg.

If the OS X dictionary (/usr/share/dict/words) does not meet your needs, you might want to generate a custom wordlist with crunch, try a wordlist containing the 500 most common passwords, or purchase Openwall's wordlist collection.

UPDATE: Forgot about crowbarKC, which has been around longer and was slightly faster than KeychainRecovery in my informal testing. John the Ripper is much faster than both, but it requires a bit more effort to install (paid binaries also available) and extract hashes. See Cracking Mac OS X Passwords and Installing John the Ripper version on Mac OSX Snow Leopard for details.

/mac | Nov 25, 2010

Installing Microsoft Office 2011 / uninstalling Office 2008 #

The Microsoft Office 2008 installer automatically searched for and offered to remove previous versions of Office. Office 2011 does not. If you'd like to cleanly uninstall Office 2008 before installing 2011, simply run "Remove Office.app" found here: /Applications/Microsoft Office 2008/Additional Tools/Remove Office/

/mac | Nov 24, 2010

Newly added to OS X page #

ThumbsUp 4.5 [512k] Batch create thumbnails via drag-and-drop  Screenshot

(via CyberNet News)

/mac | Nov 24, 2010

Delete temp files for all users #

Garrett's script Download clears the following directories for all users:
Windows XP:
%userprofile%\Local Settings\Temp
%userprofile%\Local Settings\Temporary Internet Files

Windows Vista/7:
%userprofile%\AppData\Local\Microsoft\Windows\Temporary Internet Files
However, in order for this to work on user profiles with private folders enabled, you'll need to run the script under the LocalSystem account (LSA).

/windows | Oct 24, 2010

Coworking spaces continue to grow #

From 2005:
As more and more people continue to follow their joy, leave unpleasant 9 to 5 jobs, and start their own businesses, cool spaces like theOffice and Paragraph will likely grow, providing clean, quiet environments to get work done outside of the home. Kinkos and Starbucks are swell, but they pale in comparison to the atmosphere which these places seem to offer. Rather reminiscent of the wonderful Diogenes Club.
Indeed, the desk sharing / coworking trend has grown apace; it is now easier than ever to find such havens thanks to sites like loosecubes ("rent a desk or a meeting room for a day, week, or month") and Desks Near Me ("helping freelancers get out of the house").

/misc | Oct 17, 2010

Random Kindle notes #

/eink | Oct 17, 2010

Batch create Kindle collections from document folders #

MobileRead user "collection" created a Python script (KindleCollections.py AKA KindleForResearch.py Download) to turn directories of PDFs into collections. It worked equally well for a bunch of TXT files (actually HTML files renamed due to Kindle restrictions) by simply replacing every occurrence of "pdf" with "txt" in the script.

/eink | Oct 17, 2010

Disable automatic restart after Windows 7 updates #

Open gpedit.msc and enable Computer Configuration/Administrative Templates/Windows Components/Windows Update/No auto-restart for scheduled Automatic Update installation


add NoAutoRebootWithLoggedOnUsers DWORD value to HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\Windows\WindowsUpdate\AU and assign value data of 1.

More info: How to configure automatic updates by using Group Policy or registry settings

/windows | Oct 16, 2010

The definitive guide for Linux command line newbies #

The Linux Command Line by William E. Shotts, Jr. is a clean, thorough introduction to Linux shell commands and scripts. Download the book for free and you may soon find yourself purchasing a printed copy (I did).

/nix | Oct 10, 2010

An orrery for your pocket, projector, wrist, or table #

Traveling through the solar system is easier with an orrery:

Solar Walk (iPhone/iPod Touch/iPad)
Orrery for iOS

Seeker (Mac/Win)
Orrery for OS X or Windows

Christiaan van der Klaauw's Astronomical Watches
Orrery wrist watch by Christiaan van der Klaauw

Brian Greig, orrery maker
Orrery designed by Brian Greig

UPDATE: More tools for navigating the solar system:

The mini3 original orrery kit by TheOrreryShop is made of wood, brass and acrylic.
mini3 original orrery kit

Orrery is an old-fashioned brass orrery for Windows, OS X, iOS, and Android.

Emerald Observatory for iPad displays a wealth of astronomical and chronographic information.
Emerald Observatory for iPad

The Orrery Kit is built from a number of sheets of heavy-duty card.
Orrery Kit

dD Orrery by Piotr Kaczmarek; available as a Flash application or screensaver for Mac/Win
dD Orrery

Solar System Wallpaper by Adam Dorman
Solar System Wallpaper

3D Solar System Web
3D Solar System Web

/misc | Oct 03, 2010

Import 3GP files into iMovie '08 #

Attempting to import 3GP video clips into iMovie '08 via "File" > "Import Movies..." fails without any messages. Dragging a 3GP file onto an event in the Event Library fails with the following message:
Import File
The file will not be imported.
It is not compatible with iMovie.
The solution is simply to change the file extension from 3GP to MP4 and then import via either method.

/mac | Oct 02, 2010

Text editing / shell access on a Kindle 3 #

Some possible alternate titles for this post: UPDATE: Installing Luigi Rizzo's Standalone Kindle Terminal

The Kindle 3's WebKit browser allows access to a host of web-based text editors (MyTextFile, vian, etc) and consoles (Ajaxterm, Anyterm, Web Console, shellinabox, etc).

Why bother? Because reading electronic text on an e-ink screen is far more pleasant than on any other device (including the Pixel Qi, which, however, is a great LCD for outdoor use).

After installing shellinabox on an Ubuntu Linux VM, shell access from the Kindle couldn't be simpler:

Shell access from Kindle 3

Since the Kindle keyboard lacks Esc and Ctrl, stick with simple text editing tools like cat and ed/ex instead of nano and vi (though there are ways to avoid the escape key in vi, including Alt shortcuts like Alt+M to jump to middle of page and switch to command mode):

ed running on the Kindle 3

ed/ex resources: It's easy to get really excited about ed:
Of course, on the system I administrate, vi is symlinked to ed. Emacs has been replaced by a shell script which 1) Generates a syslog message at level LOG_EMERG; 2) reduces the user's disk quota by 100K; and 3) RUNS ED!!!!!!
When I use an editor, I don't want eight extra KILOBYTES of worthless help screens and cursor positioning code! I just want an EDitor!! Not a "viitor". Not a "emacsitor". Those aren't even WORDS!!!! ED! ED! ED IS THE STANDARD!!!

/eink | Sep 30, 2010

Find total number of iCal events #

John Maisey kindly shared the following AppleScript to find the total number of events in iCal:
set myText to "Count of events in iCal's calendars." & return
tell application "iCal"
  repeat with myCal in calendars
    set myText to myText & return & (name of myCal) & ": " & (count events of myCal)
  end repeat
end tell
display dialog myText

/mac | Sep 01, 2010

Newly added to the Docs section: #

Mounting partitions from full disk images. Covers several methods for mounting with or without a specified offset.

/nix | Aug 29, 2010

Running Roadkil's Unstoppable Copier in Ubuntu Linux #

Roadkil's Unstoppable Copier can copy files from failing media while skipping files that are corrupt or unrecoverable. The Windows version has been listed for many years; here's how to get the Linux version working under Ubuntu 9.10:
  1. Download the Linux version
  2. Extract unstopcp from unstopcp.gz
  3. chmod +x unstopcp or from the GUI: right click unstopcp > Properties > Permissions > check "Allow executing file as a program" > Close
  4. Kubuntu users (or others running KDE) can simply double click unstopcp to launch it. GNOME users may find nothing happens after double clicking. Running unstopcp from the Terminal reveals: unstopcp: error while loading shared libraries: libqt-mt.so.3: cannot open shared object file: No such file or directory. libqt-mt is the Trolltech Qt library, necessary for running KDE3 apps.
  5. Head to System > Administration > Synaptic Package Manager, do a quick search for "libqt3-mt" and install it.
  6. Now GNOME users should be able to launch unstopcp successfully:
    Roadkil's Unstoppable Copier
  7. Notes:

/nix | Aug 24, 2010

Generate HTML index page of images or other HTML files #

(via Mike Mills)

/windows | Aug 23, 2010

Dwelling #

/misc | Aug 20, 2010

OS X: Mount disks as read only or block automounting altogether #

Aaron Burghardt's open source Disk Arbitrator provides a simple GUI for changing OS X's automount behavior. Automounting can be disabled completely (especially handy for drive imaging / data recovery via ddrescue, etc) or mounts can be forced as read-only (for forensics, etc, though a hardware write-blocker is still recommended).
Disk Arbitrator
This little app replaces the following hacks I've been using to prevent automounting of USB / Firewire drives in OS X: UPDATE 1: Aaron kindly replied to my email, pointing out that the "stop" command (since removed) isn't necessary in the fourth method above and that pausing/resuming diskarbitrationd with SIGSTOP and SIGCONT offers another option (though it too disables diskutil):
   $ sudo launchctl list | grep diskarbitrationd
   188 -   com.apple.diskarbitrationd
   $ sudo kill -SIGSTOP 188 # pauses diskarbitrationd, disabling automount
   $ sudo kill -SIGCONT 188 # resumes diskarbitrationd, reenabling automount

UPDATE 2: More on disabling Disk Arbitration from Apple Examiner.

UPDATE 3: Mount USB drives as read-only in Windows

/mac | Aug 17, 2010

Web browser security analysis #

Qualys BrowserCheck checks your browser as well as browser plugins and add-ons to identify insecure and out-of-date versions that put you at risk. Runs under Windows / Mac OS X, and detects:

/misc | Aug 14, 2010

OS X: Get OS version info in Terminal #

$ sw_vers
ProductName:    Mac OS X
ProductVersion: 10.6.4
BuildVersion:   10F569
$ sw_vers -productVersion
Especially handy when resetting a password in single user mode, since the method varies based on which version of OS X is running.


Here's another way to check; great for use on offline OS X installs:
$ grep -2 ProductVersion /Volumes/OfflineVolumeName/System/Library/CoreServices/SystemVersion.plist


/mac | Aug 14, 2010

Getting Google to index your Blosxom blog correctly #

I originally asked (and answered) this question on Doctype, and reproduce it here (with slight modifications) for other Blosxom users:

Blosxom creates multiple copies of the same entry in different directories. For example, a given entry would have: I wanted Google to prefer the first entry, since it makes finding content easier. I tried using robots.txt to exclude indexing of the topic and date indexes, but then Google ignores or cannot find the unique URLs either.

I ended up adding

<meta name="robots" content="noindex, follow">

to the blog index pages via this this one-liner

find -name index.html -print0 | xargs -0 perl -pi -e 's/<head>/<head>\n<meta name="robots" content="noindex, follow">/g'

at the end of my static publishing routine.

That way, search engines can find the canonical URLs, but will ignore all of the topic and date index pages.

UPDATE 1: Just found someone with the same issue who also solved it with "noindex, follow".
UPDATE 2: After a few weeks Google was properly indexing my site, and continues to do so after more than a month.
UPDATE 3: Another report of using robots meta tag to fix search engine indexing.

/blosxom | Aug 13, 2010

OS X: Printing in grayscale #

A quick search turned up a number of guides on how to print in greyscale under OS X, but they all entailed saving to PDF first and then opening in Preview. The following method worked directly from the print dialog in Snow Leopard (image) with a printer driver that did not include a native grayscale option:
  1. Paper Type/Quality
  2. Color Options
  3. Grayscale
  4. Black Print Cartridge Only
For easy access in the future, just click the "Presets:" pop-up menu and save as "grayscale" or what have you.

/mac | Aug 07, 2010

Free web app vulnerability scanning #

Zapotek is kindly offering free vulnerability scanning for web applications using Arachni, his open source vulnerability scanner written in Ruby and under active development. Zapotek is also the author of raw2vmdk (mentioned last month), which can mount raw images as VMDK virtual disks.

/nix | Jul 14, 2010

Cross platform testing and benchmarking suite #

The Phoronix Test Suite is a testing and benchmarking platform that runs under Linux, OpenSolaris, Mac OS X, Windows 7, and BSD (a Linux-based live DVD/USB version is also available). In addition to detailed system software and hardware information, Phoronix can report CPU temperature, battery power consumption, disk read/write speeds, etc.

/nix | Jul 09, 2010

Skype Test Call contact missing #

Skype's sound set up guide offers the following suggestion:
"To test your sound set up in any version of Skype, use the Skype Test Call robot - find it in your contact list."
Sadly, it was missing from my contact list, and searching the directory for "skype test", "skype test call", etc returned a vast number of fake test call accounts. The real Skype Test Call robot username is echo123.

/misc | Jul 09, 2010

Malware analysis and forensic investigation environments #

(via The H Security and Darknet)

/nix | Jul 09, 2010

Safari 5 hearts OpenDNS #

There is no shortage of reports on Safari 5's interminable page load times: The "fix" seems to be switching DNS servers to OpenDNS. In my own informal testing, Safari 5 was basically unusable with Google's Public DNS: pages would either refuse to load altogether, or would take much longer than usual. Simply switching to OpenDNS servers fixed the problem instantly. Switching back and forth between Google Public DNS and OpenDNS (along with testing page load times via Webmonkey Stopwatch) confirmed Safari's bias.

/mac | Jun 30, 2010

Flash Cookies #

When clearing your web browser's cache / cookies / temp files / etc, don't forget to wipe Flash cookies (*.SOL files, properly known as Local Shared Objects, or LSO) too. OS X users will find them buried within subfolders in

  ~/Library/Preferences/Macromedia/Flash Player/#SharedObjects/ and
  ~/Library/Preferences/Macromedia/Flash Player/macromedia.com/support/flashplayer/sys/

Be aware that Flash cookies are not deleted by Safari's "Reset Safari", Opera's "Delete Private Data", Firefox's "Clear Recent History", or any other browser's cleaning method. (Iain Cheyne kindly points out that the BetterPrivacy add-on for Firefox can manage Flash cookies.)

/misc | Jun 29, 2010

Smartphone remote kill switches #

or, "Why I still use a Palm Treo 755p":

/misc | Jun 26, 2010

Convert raw disk image or physical disk to virtual machine #

"Live View is a Java-based graphical forensics tool that creates a VMware virtual machine out of a raw (dd-style) disk image or physical disk. This allows the forensic examiner to 'boot up' the image or disk and gain an interactive, user-level perspective of the environment, all without modifying the underlying image or disk. Because all changes made to the disk are written to a separate file, the examiner can instantly revert all of his or her changes back to the original pristine state of the disk. The end result is that one need not create extra 'throw away' copies of the disk or image to create the virtual machine.

Live View is capable of booting Containing the following operating systems Behind the scenes, Live View automates a wide array of technical tasks. Some of these include: resolving hardware conflicts resulting from booting on hardware other than that on which the OS was originally installed; creating a customized MBR for partition-only images; and correctly specifying a virtual disk to match the original image or physical disk."
Related projects: UPDATE: In my testing, Live View 0.7b / VMWare Server 1.x (2.x not supported by Live View) on a Windows 7 64-bit host did not work, even after trying a number of suggestions in the forums.

Switched to a Windows XP host, and successfully tested with a raw disk image of an XP install. However, attempting to use a partition image instead of a full disk image resulted in the following error on boot:
   A disk read error occurred
   Press Ctrl+Alt+Del to restart
Apparently the MBR is not being written correctly, or perhaps VMWare just doesn't play well with images created from partitions.

Finally, I tried using an IDE drive (Windows XP installed) with two different USB and Firewire bridges, but that also failed:
   ERROR>     Snapshot Creation Failed
   ERROR>     Problem preparing partition1 for launch
   VM Launch Failed
tl;dr: Windows XP is well-supported as a host, but you may not be able to work with anything other than full disk images.

/windows | Jun 18, 2010

Generate wordlists with crunch #

$ wget http://downloads.sourceforge.net/project/crunch-wordlist/crunch-wordlist/crunch2.4.tgz
$ tar -xvf crunch2.4.tgz
$ cd crunch

-bash: cd: crunch: Permission denied

Let's check the file permissions:

$ stat -c %a crunch

Ah! Just need to fix that:

$ chmod 755 crunch
$ cd crunch
$ make

Now we're ready to generate our list. In this example, we'll make a list consisting of every possible combination of 3 letters (both upper and lower case) followed by 2 numbers:

$ ./crunch 5 5 -f charset.lst mixalpha -o list -t @@@%%

Even though we've specified mixalpha as the character set, crunch replaces the percent symbols with numbers. This 80MB, 14060800-line list only took crunch 5 seconds to generate.

/nix | Jun 13, 2010

The Peaceful Mind #

/misc | Jun 05, 2010

POP, IMAP, and Exchange client for Android #

K-9 Mail is an open source email client for Android with support for POP, IMAP, and Exchange. K9 has two major benefits over Android's built-in Email app:
  1. It works
  2. Option to Bcc a copy of sent mail (ideal for POP accounts)

/misc | May 25, 2010

Newly listed full-screen text editors #

Zeus Poplar kindly sends word of Q10, a full-screen portable text editor. Unlike the previously mentioned WriteMonkey, Q10 does not require .NET.

Green Award Q10 1.2.21 [357k] + Full-screen, portable text editor  Screenshot

UPDATE: Just happened upon another standalone distraction-free text editor on Ghacks:

Green Award Zenedit 1.4 [283k] + Minimalist text editor designed to prevent distractions  Screenshot

/windows | May 23, 2010

Windows Firewall alternative #

Newly added to the Internet page:

GhostWall 1.150 [656k] Windows Firewall alternative ideal for low latency applications.  Screenshot

/windows | May 23, 2010

Ubuntu 10.04: Bypass minimum password length / complexity requirements #

Ubuntu 10.04 is requiring users to create more secure passwords than in previous versions. That is generally a good idea, but there are cases where you might want a simple password (for testing purposes in a VM, etc). The fastest method to bypass the minimum complexity and length requirements is to simply run the passwd command as root:

$ sudo passwd username

I mucked around in /etc/pam.d/common-password and read the man page for pam_unix before just chucking it and going with the above workaround.

/nix | May 18, 2010

Blosxom.com and Blosxom plugin registry listing mirror #

After reading this, I decided to post an archive of Blosxom.com and the Version 2 Plugin Registry Listing, including the code for all 465 plugins (602 counting the various versions):

Those desiring to dig deeper into Blosxom may enjoy Frank Hecker's annotated blosxom.cgi and Rob Reed's annotated Blosxom bundle.

/blosxom | May 14, 2010

Send To SFTP for Mac OS X #

There have been a number of Send To FTP/SFTP apps for Windows, but Nick Jensen's OneWay is the first OS X option I've seen. It allows quick and easy (S)FTP uploads via the Context Menu:

OneWay - SFTP uploads via the Context Menu

(via Hacker News)

/mac | May 12, 2010

Free PDF to DOC converters #

freewaregenius.com has a detailed review of free online and offline PDF to DOC converters. In my brief and unscientific testing, SmartSoft's lightweight Free PDF to Word Converter was the best for converting a long text-based PDF.

/misc | May 10, 2010

Delete / remove blank pages from a PDF #

The following was originally posted to http://www.accesspdf.com/article.php/20050128092744804. Found a copy on the Wayback Machine, which is apparently not indexed by Google.

Removing Blank Pages from a PDF

Friday, January 28 2005 @ 09:27 AM PST
Contributed by: Admin

Here is an idea for how to remove blank pages from a PDF using pdftotext and pdftk. It is based on a recent posting to comp.text.pdf.

demunn@yahoo.com wrote:
> Hello all,

> Sorry if this is a recurrent question. I'm rendering/printing an HTML
>  document from a web-based program to a pdf file. The web-based
> program has minimal features to control pagination, etc. (I know
> web-based print control is relatively primitive) and the outcome is
> unwanted blank pages in the PDF output file.

> Anyway, I'm basically looking for a program that will allow me batch 
> process a folder of PDF files and strip out the blank pages. Is there
>  any programs or utilities that will do this? Any suggestions are
> greatly appreciated

> DM

Here is one simple idea that assumes that all non-blank pages have
(extractable) text on them.

1. Use pdftotext (from the xpdf project) to convert mydoc.pdf to
mydoc.txt. Pdftotext uses the formfeed character (0x0c) to mark page breaks.

2. Scan mydoc.txt looking for pages with no text. Record these page
indexes (start counting at page 1, not zero).

3. If you find blank pages, use pdftk to remove them. Construct the
pdftk command line using the page indexes you collected in step 2. For
example, to drop page 3, say:

   pdftk mydoc.pdf cat 1-2 4-end output mydoc.noblanks.pdf

It shouldn't be too hard to write such a shell script, eh?

Sid Steward

The Script

Using bash (via MSYS) on my Win2k machine, I have strung some commands together that identify PDF pages with no extractable text on them. I don't say "blank pages," because sometimes a non-blank PDF page has no extractable text on it.

# find_textless_pdf_pages.sh
# bash script for MSYS; also requires pdftotext (xpdf);
# identify PDF pages that have no extractable text on them;
# linux users might need to omit the -c sed option and then
# drop the 'R' from the sed script;
# invoke like so:
#  find_textless_pdf_pages.sh mydoc.pdf
pdftotext $1 - |
tr "FRfr" "frFR" |
sed -c -n '/^FR$/{ N; /^FRnFR$/a
D; }'

Command Breakdown

pdftotext - converts the input PDF file into text. It uses the formfeed character (f) to mark page breaks.

tr - translates characters. Sed doesn't see non-printing ASCII characters such as f or r (carriage return). So, translate R->r, F->f, and f->F, r->R.

sed - the stream editor. I discuss sed here. If it finds a line that is just "FR", then it looks ahead to see if the next line is also "FR". If it is, then it prints "PageNoText". If it isn't, then it prints "Page". Finally, it uses the D command to continue processing with just the second line of text.

I'll continue working on this script as time permits.

/nix | May 10, 2010

Fujitsu ScanSnap S1300 vs S1500 #

I was debating between the ScanSnap S1300 and S1500 until watching this video. Debate over. For the tl;dw crowd: the S1500 scanned 15 double-sided pages in 30 seconds, while the S1300 took 107 seconds.

/misc | May 10, 2010

Display atime, ctime, and mtime for a file #

$ stat foo
  File: foo
  Size: 221       	Blocks: 8          IO Block: 4096   regular file
Device: 801h/2049d	Inode: 540949      Links: 1
Access: (0700/-rwx------)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2006-12-04 07:00:00.000000000 -1000
Modify: 2006-12-04 07:00:00.000000000 -1000
Change: 2010-05-08 10:01:29.000000000 -1000

stat also accepts wildcards: $ stat *.txt

/nix | May 08, 2010

Boot any and all ISO images from USB drive #

I spent days looking for a way to boot any and all ISO images from a USB drive. I envisioned being able to simply copy any bootable CD or DVD (Windows installers, OS X boot discs, Linux Live CDs and recovery discs like Knoppix, etc) to a USB drive and boot it normally. Some of the possible solutions I found included:

The more robust solutions were generally quite tedious, and none of them worked for all ISO images, as explained here: Boot "any" ISO image or boot "all" ISO images.

Having recently stumbled upon a floppy drive emulator, I started looking for a USB CD/DVD drive emulator and finally found a way to boot any ISO image from USB drive: the iodd 2501 (newer Zalman models available - see update 5 below). It is a 2.5" SATA drive enclosure with an LED display to select the desired ISO image. Simply create a "_ISO" directory in the drive root and dump your ISO images inside. That's it! The enclosure also has a write protect switch and eSATA support.

The iodd 2501 has booted every ISO image I have thrown at it, on both PC and Mac hardware. When I first tried using the device, the LED would only report "NO-_ISO", despite having followed the instructions exactly. It turns out that formatting as FAT32 in OS X's Disk Utility was the problem; I simply used fat32format under Windows and the problem was solved (NTFS and exFAT are supported with a firmware upgrade).

I purchased mine from LinITX; they've got detailed instructions, videos, and great customer service.


  1. John Rothlisberger wrote in to ask if ISO images can be changed on the fly during an install. After testing with a Windows XP Media Center Edition 2 disc set I am happy to report that they can.
  2. Curtis Reno has started a neat project to build an open source USB CD emulator using inexpensive components. Announcement | Fourm
  3. isostick is a similar CD emulator that uses microSD cards and a USB Flash drive adapter. Sadly, it only supports FAT32, though large ISO files can be split.
  4. Finally got around to updating the firmware on my original iodd 2501. Since the English support site is gone now, here are the necessary files and instructions in case anyone else needs them (current as of May 2013).
  5. Zalman (which took over iodd development) continues to improve the hardware, recently adding hardware encryption and more. Current models include the ZM-VE400, ZM-VE300, ZM-VE200 SE, and ZM-VE200.
  6. The only problem I've encountered with these virtual ODDs is booting some Dell models when a physical ODD is present; simply unplug the physical ODD to circumvent.

/misc | May 08, 2010

Knoppix 6: Mount NTFS drive as writable #

  1. If you've already clicked on the NTFS device icon within PCMan File Manager, right click the icon again and select "Unmount File System" (since Knoppix mounts the drive as read-only from the GUI).
  2. Launch Terminal Emulator and run fdisk to identify the NTFS partition:
    $ sudo fdisk -l
  3. Mount the device (/dev/sda1 in this example) as writable:
    $ sudo ntfs-3g /dev/sda1 /mnt/sda1
  4. When finished, be sure to unmount:
    $ sudo umount /mnt/sda1

/nix | May 01, 2010

Safari 4: Bookmark all tabs in all windows #

Sometimes you might end up with 4 or 5 Safari windows, each with 10 or 15 tabs. In order to bookmark and save them all into a single new folder, simply click Window > Merge All Windows > Add Bookmark for These n Tabs...

Mac OS X Hints hosts several AppleScripts for bookmarking all tabs as well.

/mac | May 01, 2010

Randomize lines with custom delimiter #

Arthur de Jong's rl (randomize lines) "reads lines from an input file or stdin, randomizes the lines and outputs a specified number of lines. It does this with only a single pass over the input while trying to use as little memory as possible." It came in handy when I wanted to randomize a collection of quotes which looked like this:

Nothing, to my way of thinking, is a better proof of a well ordered mind than a man's ability to stop just where he is and pass some time in his own company.



There is an old story about a famous rabbi living in Europe who was visited one day by a man who had traveled by ship from New York to see him. The man came to the great rabbi's dwelling, a large house on a street in a European city, and was directed to the rabbi's room, which was in the attic. He entered to find the master living in a room with a bed, a chair, and a few books. The man expected much more. After greetings, he asked, "Rabbi, where are your things?" The rabbi asked in return, "Well, where are yours?" His visitor replied, "But, Rabbi, I'm only passing though," and the master answered, "So am I, so am I."

Jack Kornfield


Simplicity of life, even the barest, is not a misery, but the very foundation of refinement.

William Morris

Here's how I randomly reordered them with rl, specifying "%" as the delimiter instead of the default newline:

$ rl -d% quotes.txt > randomized_quotes.txt

The latest version is 0.2.7. Install via ./configure && make && make install, or OS X users can grab this binary I compiled under 10.6.

(tags: unix, randomize, lines, text, reorder, rearrange)

/nix | Apr 29, 2010

Transfer large files directly between computers on the Net #

There are a number of free, third-party solutions for transferring files directly from computer A to computer B across the Internet: However, these services have several issues: Here is a quick and easy alternative for Mac users which involves no signups or third parties:
  1. On server: System Preferences > Sharing > check "Remote Login"
  2. On server router: Forward port 22 to server
  3. On client: Use a SFTP client (either the included CLI sftp or a GUI-based app like Transmit, CyberDuck, YummyFTP, etc) to connect to the external IP address of the server - instant file sharing!
Opening ports and enabling services can expose your network to additional risk. See also: Hardening SSH and Mounting Remote Filesystem in OS X Finder via SSHFS.

/mac | Apr 28, 2010

Shrink PDF file sizes with ShrinkIt #

ShrinkIt 1.1 [181K] Strips needless metadata from PDFs to reduce file size.  Screenshot

/mac | Apr 27, 2010

The disk cannot be partitioned because some files cannot be moved. #

Boot Camp offers some really bad advice when it can't partition the disk:
The disk cannot be partitioned because some files cannot be moved.
Back up the disk and use Disk Utility to format it as a single Mac OS Extended (Journaled) volume. Restore your information to the disk and try using Boot Camp Assistant again.
A much more helpful message would be:
You've got some big file(s) I can't move. Find 'em (find ./ -size +100M) and get 'em off the disk, or just use iDefrag instead.
No, actually this is what the message should read:
You need to move big_file.iso to another disk before Boot Camp setup can continue.

/mac | Apr 27, 2010

Hard drive enclosure with write protect switch #

It is getting harder and harder to find USB flash drives with write protect switches. Thankfully, ACARD Technology makes the ACP-2127, a 2.5" USB to SATA hard disk enclosure with hardware write protect switch. Paired with a fast HDD or SSD, this could be the perfect replacement for a technician's aging flash drive. Mars Tech has the best price ($20), and awesome phone-based customer service to boot.

/misc | Apr 26, 2010

Newly-added hex editor #

Green Award HxD [789k] + Hex editor / disk editor / RAM editor with support for huge file sizes ("if it fits on a disk/drive you can open it"). Features include: fast find and replace, checksum calculation, file shredding, concatenation or splitting of files, statistics, and more.  Screenshot

/windows | Apr 24, 2010

Oh, you didn't need that boot loader, did you? #

It's been years since the outcry over TurboTax writing to the boot track forced them to drop DRM altogether. Inexplicably, AutoCAD continues to use this dangerous copy protection scheme (Macrovision FLEXnet or SafeCast or whatever it's called), writing to sector 32 of the boot track. As ExtremeTech pointed out long ago:
Unfortunately, these "reserved" sectors of the hard drive aren't necessarily a safe place for data. And they're an especially dicey place to keep licensing information ... Data compression utilities, "multiboot" utilities, password protection and encryption software, and sector translation software (which allows older computer systems to accept today's huge hard drives) may also reside in this area.
According to reports, Macrovision DRM has exposed customers to malware attacks, caused TrueCrypt volumes to become unbootable, and destroyed non-Windows partitions. To top it all off, SafeCast is apparently easily circumvented by the "bad guys" anyway.

Autodesk, a humble request: please refrain from making potentially dangerous, low-level changes to your customers' hard drives. It really ruined an otherwise perfectly good week for a friend of mine.

/windows | Apr 24, 2010

Open source hex editor for OS X #

Hex Fiend 2.0 [935k] {S} Hex editor with support for huge (over 100GB) files.  Screenshot

/mac | Apr 22, 2010

Full-screen, portable text editor #

WriteMonkey is a full-screen, portable plain text editor with: customizable colors, fonts, and sounds, statistics, bookmarks, text replacement shortcuts, autosave, history, Markdown/Textile shortcuts and export, and much more. Requires .NET Framework. (via Download Squad)

/windows | Apr 22, 2010

Excellent freeware firewall for Vista and Windows 7 #

Windows 7 Firewall Control 3.5 weighs in under 1.3MB, comes in both 32 and 64 bit versions, and has a portable version to boot! Runs fast and clean, with very little impact on system performance. Offers ingress (inbound) and egress (outbound) filtering. Brings back memories of ZoneAlarm's early, pre-bloat years. Shareware version also available.

/windows | Apr 21, 2010

Changing / spoofing your MAC address in Windows and OS X #

/misc | Apr 21, 2010

4DOS > 4NT > TCC LE #

Take Command Console LE (TCC LE) is a freely available version of the command processor formerly known as 4NT (and before that, 4DOS). It's like cmd.exe on steroids, with command history, built-in file viewer, core scripting language, hundreds of new command options, and much more.

/windows | Apr 21, 2010

OS X Address Book contacts not syncing with iPhone #

mont3's instructions from 2008 helped us resolve a local (USB) Address Book sync issue. I have fixed a number of typographical and procedural errors (including the potentially tragic instructions to "Go to FINDER, LIBRARY, APPLICATION SUPPORT and delete everything in there") and updated the process for Snow Leopard:
  1. Backup. Backup. Backup.
  2. Export all contacts from the OS X Address Book as a vCard (attempting to use an Address Book Archive instead did not resolve the sync issue). Note that groups are lost during this process.
  3. Delete all contacts and groups from the OS X Address Book and then quit Address Book.
  4. Delete ~/Library/Caches/com.apple.Address Book
  5. Delete ~/Library/Preferences/com.apple.AddressBook.*
  6. Delete all files and folders inside of ~/Library/Application Support/AddressBook. Note that images are lost during this process. See mont3's original post for information on saving them first.
  7. Open /Applications/iSync.app and click iSync > Preferences > Reset Sync History... Reset Sync History.
  8. Import the vCard containing all of your contacts into Address Book.
  9. While holding the Command and Option keys (to prevent auto sync), plug in your iPhone. If "iTunes is running in safe mode" message appears, click "Continue" while continuing to hold the Command and Option keys.
  10. Click the iPhone icon under "Devices" and then click the "Info" tab.
  11. Make sure "Sync Address Book contacts" is set to sync "All contacts". For good measure you may want to select "Contacts" under "Replace information on this iPod:" if you don't mind having the Mac overwrite the iPhone on the next sync only.
  12. Click "Sync".

/mac | Apr 21, 2010

Migrate from Windows Mail to Apple Mail #

If you're moving from Vista's Windows Mail to OS X's Mail, look no further than Jürgen Lüthje's eml2mbx to convert your mail folders / eml files to mbox format. Or go the other way with his mbx2eml. Both apps are rather reminiscent of Ulrich Krebs' DbxConv, which converts Outlook Express dbx files to mbox or eml.

UPDATE: See eml to mbx converters

/windows | Apr 21, 2010

Extract strings from raw disk device or image #

After filling a hard drive with zeroes (followed by a quick format), I wanted to ensure that no sensitive data remained. My first thought was to use a disk editor/viewer like iBored or Disk Investigator, but scrolling through millions of blocks gets dull pretty quickly. So I tried piping dd to strings - bingo:
$ sudo dd if=/dev/rdisk2 bs=512 | strings -a
BSD  4.4
pEFI        FAT32   
Non-system disk
Press any key to reboot
Initially, I used sudo kill -s SIGINFO dd_pid to check on dd's progress, but then remembered dcfldd, an enhanced version of dd with a much better progress indicator (among many other improvements):
$ sudo dcfldd if=/dev/rdisk2 bs=512 | strings -a
6144 blocks (3Mb) written.EFI        (
124928 blocks (61Mb) written.
UPDATE: While looking for a hex editor that would search for non-matching patterns, I stumbled upon an even simpler solution: use hexdump or od (syntax is the same for both):
$ hexdump /dev/rdisk2
0000000 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Duplicate lines are truncated (as indicated by the asterisk) unless the -v option is specified:
$ od -v /dev/rdisk2
0000000 0000 0000 0000 0000 0000 0000 0000 0000
0000010 0000 0000 0000 0000 0000 0000 0000 0000
0000020 0000 0000 0000 0000 0000 0000 0000 0000

/mac | Apr 14, 2010

Computer simulators and assembly language learning tools #

6502 is an integrated macro assembler, simulator and debugger for 650x microprocessor family. Lets you write, test and run 6502/65c02 programs

emu8086Assembler Visual Builder is a web-based Java applet that allows you to create an assembly language program and to simulate running the program. The simulator shows you the machine cycle - the fetch, decode, execute steps - as each assembly instruction is executed.

CPU Sim is an interactive Java-based CPU simulator for use in introductory computer organization classes. It allows users to design simple computer CPU's at the microcode level and to run machine-language or assembly-language programs on those CPU's through simulation. It can be used to simulate a variety of architectures, including accumulator-based, RISC-like, or stack-based (such as the JVM) architectures.

EASy68K is a 68000 Structured Assembly Language IDE. EASy68K allows you to edit, assemble and run 68000 programs on a Windows PC or Wine.

emu8086 is an 8086 microprocessor emulator with integrated assembler and tutorials for beginners. The emulator runs programs like the real microprocessor in step-by-step mode. It shows registers, memory, stack, variables and flags. All memory values can be investigated and edited by a double click. The instructions can be executed back and forward. emu8086 can create a tiny operating system and write its binary code to a bootable floppy disk. The software package includes several external virtual devices: robot, stepper motor, led display, and traffic lights intersection. Additional devices can be created.

Lia is a simple computer simulator and assembly editor written in pure Java. It enables you to write/edit/save your "assembly" files and then execute the code using the built-in simulator. A list representing the memory contents is available at all times so the programmer can see what is really happening. Full source code included.

TOM Computer SimulatorLogicSim allows you to design and simulate digital logic circuits with logic gates like AND, OR, FlipFlop, etc.

Logisim is an educational tool for designing and simulating digital logic circuits. Circuit layouts can be used as "subcircuits" of other circuits, allowing for hierarchical circuit design. (via David Gould)

MARS (MIPS Assembler and Runtime Simulator) is a lightweight interactive development environment (IDE) for programming in MIPS assembly language, intended for educational-level use with Patterson and Hennessy's Computer Organization and Design.

SPIM is a self-contained simulator that will run MIPS32 assembly language programs. It reads and executes assembly language programs written for this processor. SPIM also provides a simple debugger and minimal set of operating system services. SPIM does not execute binary (compiled) programs.

TOM Computer Simulator is designed for computer architecture and programming courses. TOM stands for Thoroughly Obedient Moron, and consists of a number of components each designed to teach the fundamentals of computing via simple interactions.

/misc | Apr 14, 2010

Memory lane #

Read Creative Computing, Compute!, and more at The Classic Computer Magazine Archive.

/misc | Apr 12, 2010

Grep is greedy; make it less so #

While editing an HTML file in TextWrangler, I needed to find and replace all occurrences of:
<span class="a">text</span>
This regex:
\<span class=\"a\"\>.*\<\/span\>
returned results like:
<span class="a">text</span><span class="b">more text</span>
Using a non-greedy quantifier (the question mark) returned the desired results:
\<span class=\"a\"\>.*?\<\/span\>

UPDATE 1: Wikipedia has more on lazy quantification: "... modern regular expression tools allow a quantifier to be specified as lazy (also known as non-greedy, reluctant, minimal, or ungreedy) by putting a question mark after the quantifier (e.g., <.*?>) ..."

UPDATE 2: To match multiline patterns in TextWrangler's grep, start the expression with (?s). According to the BBEdit-TextWrangler Regular Expression Cheat-Sheet m (multiline) "allows the grep engine to match at ^ and $ after and before at \r or \n" and s (magic dot) "allows . to match \r and \n".

/mac | Apr 11, 2010

Batch download YouTube videos from the command line #

UPDATE: See also Download videos from YouTube, Metacafe, Daily Motion, Vimeo, and more.

youtube-dl 2010.04.04: Written in Python; public domain license; excellent documentation; runs under *nix, OS X, Windows; works with latest YouTube changes. First attempt to batch download in bash returned an error:

$ youtube-dl.sh -o %(title)s.%(ext)s --batch-file urls.txt --best-quality --no-overwrites
-bash: syntax error near unexpected token `('
Just had to enclose the output template in single quotation marks:
$ youtube-dl.sh -o '%(title)s.%(ext)s' --batch-file urls.txt --best-quality --no-overwrites
The URLs for urls.txt were collected from a YouTube playlist webpage like so:
$ grep -o '/watch?v=.\{11\}' source.htm | sed 's/^/http:\/\/youtube.com/' | uniq > urls.txt
with sed prepending http://youtube.com to grep's output, and uniq culling the duplicates.

UPDATE 1: To specify the desired quality format, use -f #, where # is one of the format values shown in the first row below:

fmt value 5 6 34 35 18 22 37 38 83 82 85 84 43 44 45 46 100 101 46 102 13 17
Default container FLV MP4 WebM 3GP
Video Encoding Sorenson H.263 MPEG-4 AVC (H.264) VP8 MPEG-4 Visual
Profile Main Baseline High 3D 3D
Resolution progressive 224p 270p 360p 480p 360p 720p 1080p 2304p 240p 360p 520p 720p 360p 480p 720p 1080p 360p 480p 540p 720p
Max width (pixels) 400 480 640 854 640 1280 1920 4096 854 640 1920 1280 640 854 1280 1920 640 854 1920 1280 176
Max height (pixels) 240 270 360 480 360 720 1080 3072 240 360 520 720 360 480 720 1080 360 480 540 720 144
Bitrate (Mbit/s) 0.25 0.8 0.5 0.8–1 0.5 2–2.9 3–4.3 3.5–5 0.5 2-2.9 0.5 1 2 0.5 2
Audio Encoding MP3 AAC Vorbis AAC
Channels 1–2 2 (stereo) 1
Sampling rate (Hz) 22050 44100 22050
Bitrate (kbit/s) 64 128 96 152 96 152 128 192 128 192

(Above table is from Wikipedia's YouTube entry)

UPDATE 2: Video Backup Fusion offers an easy way to save all upload or playlist URLs to a text file (the download function seems to have broken as of May 2014). Source code and binaries available for Windows, Linux, and Haiku. You may want to uncheck "Show video thumbnails" in Settings and change the "Max. videos in list" value to 5000 (the maximum it will accept). Highlight all results, right click, select "Copy url(s)", and paste into your favorite text editor.

UPDATE 3: It's been almost 5 years since this post, but youtube-dl is still going strong, having just been updated yesterday (2015-03-28). Here are two more neat ideas from a HN thread:

/nix | Apr 11, 2010

Unpack / extract .MSI file contents #

MSI Unpacker extracts files from .MSI installer packages. I used it (along with Universal Extractor) to unpack the Debugging Tools for Windows (required for using Crash Reporter in ERD Commander 2005) as follows:
  1. Downloaded dbg_x86_6.7.05.1.exe
  2. Dropped dbg_x86_6.7.05.1.exe onto UniExtract.exe, which extracted dbg_x86.msi
  3. Dropped dbg_x86.msi onto MSIUnPack.vbs, which extracted the entire Debugging Tools for Windows directory
(Note: While UniExtract does have limited MSI support, it was unable to unpack dbg_x86.msi.)

/windows | Apr 07, 2010

Best organic Japanese tea source on the Net #

Just received our first order of matcha from Yuuki-Cha.com - very impressed with their fast service and exceptional tea. In addition to matcha, they carry sencha, gyokuro, genmaicha, etc, all of which are certified organic by JAS; some also carry USDA organic certification. If you're into Japanese tea, check them out.

/misc | Apr 01, 2010

Tiny command-line disk/partition apps for DOS/Windows #

For over ten years, Svend Olaf Mikkelsen has crafted a number of free, tiny CLI apps focused on partitions and data recovery, like:

/windows | Mar 31, 2010

Fix your Kindle's tiny fonts #

Kindle Optimizer - Digital imaging veteran fixes fonts for Amazon: Kindle 2.0.2 and 2.0.3 files | Kindle 2.3 files

/eink | Mar 31, 2010

Cropping PDFs #

Three scripts for cropping PDFs (especially useful for making PDFs easier to read on a Kindle): UPDATE: Just stumbled onto soPDF, a Windows app (Linux port available) that is very fast and effective at removing white space:
$ sopdf -i input.pdf -o output.pdf -m 2

Before and after screenshots via PDFCrop:


/nix | Mar 31, 2010

Remove PDF copy/print restrictions #

Free PDF Unlock Online Utility (Beta) quickly and easily unlocks PDFs up to 5MBs.

/misc | Mar 30, 2010

Store and read HTML files on the Kindle 2 #

UPDATE: Sorry, just realized that this is essentially a dupe of Reading HTML files on a Kindle from last year.

The long (and battery-draining) way:
  1. Copy .html files to a new top level directory, e.g., "html"
  2. Launch Basic Web (wireless must be enabled) and enter URL like file:///mnt/us/html/file.html
The short way:
  1. Change file extension from .html to .txt before copying to "documents" on the Kindle
  2. That's it! Just open your file from the Kindle home screen - basic formatting (h1, p, i, etc) is preserved.
(via Tomster)

/eink | Mar 27, 2010

Rob Reed builds a better Blosxom with Ode #

I first contacted Rob Reed last year while looking for a copy of his annotated Blosxom bundle (not to be confused with Frank Hecker's version). Rob quickly and kindly obliged, and also informed me of a new publishing platform he was working on called Ode, Release 1.0 has just become available for download today.

I asked Rob for permission (which was graciously granted) to post some of the thoughts he has shared over the last few months via email (emphasis mine):

/blosxom | Mar 25, 2010

Computing like it's 1989 #

Zach Perry kindly informs us of his recently launched Alternative Electronic Information Network (AEIN), a fee-based BBS that requires a proprietary Windows or DeskMate app to access. Normally such a service would not be mentioned in this space (especially with free and open options like Uncensored available), but the retro ANSI art and whimsical list of free additional benefits (DOS/Windows tech support by email, smoke alarm batteries, camera and computer parts, tree planting donation, etc) tipped the scales. Excuse me while I go grab a copy of TheDraw.

UPDATE: Just noticed this announcement on the homepage a week later:
The AEIN is now transitioning to a free service. Members who purchased paid memberships will retain the benefits promised at the time they signed up. New users can click here to download the AEIN software. All extra benefits have been eliminated for new members, but users can still access a wide range of ad-free content; news, weather, recipes, reviews, computer tips, world time, technical information, etc. New development has come to an end, although news and weather updates will continue.

/windows | Mar 22, 2010

iText Express 3.4: TextEdit on steroids #

Japanese | English. 32 and 64 bit versions available, supports Panther through Snow Leopard, optional OgreKit plugin greatly improves search function. 3.3 review. Screenshot of ame ni mo makezu pasted into one of the included templates:

/mac | Mar 20, 2010

Automatically adjust color temperature of display based on time of day #

f.lux "makes your computer screen look like the room you're in, all the time. When the sun sets, it makes your computer look like your indoor lights. In the morning, it makes things look like sunlight again. Tell f.lux what kind of lighting you have, and where you live. Then forget about it. F.lux will do the rest, automatically."

Certainly seems to make backlit LCDs easier on the eyes in my testing. This should hold me over until Pixel Qi releases their DIY kit. Windows, Mac, and Linux versions available.

/misc | Mar 12, 2010

Random Data File Creator #

We've covered how to create empty files in Windows with fsutil. Random Data File Creator (RDFC) allows you to create binary files of any size filled with random numbers.

/windows | Mar 07, 2010

An embarrassment of riches #

Claus recently mentioned ImDisk Virtual Disk Driver and its author Olof Lagerkvist in such glowing terms that I had to have a look. In short, I was floored. Olof has crafted dozens of open source tiny apps, virtually all of which run under Windows 95 through 7 (some even under Windows 3.x!). Even better, he is one of the few Microsoft Visual C++ developers who shuns msvcrt.dll:
These binaries are compiled with the free version of Microsoft Visual C++. They are not linked with the standard C or C++ libraries that comes with this compiler, instead they are linked with a dynamic library I call minwcrt (Minimal Windows C Run-Time Library) linking them to crtdll.dll instead of msvcrt.dll. This makes them run in all versions of Win32 without additional dll files. This is also the reason why the exe files are very small. Source for this library is available as part of the source archive mentioned in the top of this document.
Here is just a smattering of the many apps available:

/windows | Mar 07, 2010

OS X's popup Dictionary panel and Japanese #

Command+Control+D while hovering over a word displays a popup Dictionary panel in OS X. Pronunciation for English words is displayed in the medium grey title bar, but not for Japanese:
Anyone know of a fix or workaround? In the meantime, JEDict's web browser (built on WebKit) includes a drawer with pronunciation, definitions, etc.

/mac | Mar 07, 2010

Automatically insert furigana / ruby / rubi readings in Firefox #

Furigana Injector looks up the readings for kanji words and inserts them as furigana. Optionally install XHTML Ruby Support for rendering furigana above text instead of inline.

/misc | Mar 07, 2010

Gitso (remote support for Windows, Linux, OS X) updated to 0.6 #

Aaron Gerber kindly informs us of an update to the insanely awesome, open source, cross-platform remote support app that he and Derek Buranen (along with several other contributors) have continued to craft over the past several years, Gitso:

/misc | Feb 21, 2010

Tiny HDD activity monitors #

Especially useful for computers without HDD LEDs or for monitoring hard drive activity on remote machines: All five have been bundled and uploaded here for posterity's sake.

/windows | Feb 21, 2010

Batch crop images with GIMP and ImageMagick #

  1. Backup your images!
  2. With GIMP's Rectangle Select tool, highlight area to crop and note values for Position and Size:

    Position: 940, 645 | Size: 93, 30
  3. $ mogrify -crop 93x30+940+645 *.png

/nix | Feb 21, 2010

Connecting to Time Capsule from Windows 7 #

When connecting to a Time Capsule network share from Windows 7, you are prompted for a username and password. The username is "admin" and the password is (by default) whatever password you assigned your Time Capsule during setup. However, you will not be able to login without tweaking some security settings (just like this tip for Vista from 2007):

Start > Search box > secpol.msc > Local Policies > Security Options > "Network security: LAN Manager authentication level" > set to "Send LM & NTLM - use NTLMv2 session security if negotiated".

You may also need to uncheck both boxes under "Network security: Minimum session security for NTLM SSP based (including secure RPC) clients".

/windows | Feb 16, 2010

Using HFSExplorer under Windows 7 x64 #

HFSExplorer (a free alternative to MacDrive for reading Mac-formatted hard disks and disk images) would not detect HFS+ drives in Windows 7 x64 until changing the compatibility mode as follows: right click hfsexplorer.exe > Properties > Compatibility > check "Run this program in compatibility more for:" > select "Windows XP (Service Pack 2)" > click OK.

/windows | Feb 16, 2010

Back4Sure #

Ulrich Krebs, author of DbxConv (featured back in 2008), has crafted a simple backup program with one key feature missing from so many others: the ability to specify a target volume label instead of drive letter. Those who have struggled with trying to assign the same static drive letter to multiple devices in Windows will understand how helpful this is. My only suggestion for version 2 would be enabling backup of in-use files. Freeware for personal or business use, though there is an occasional donation nag screen.

Green Award Back4Sure v1 [694k] + Simple backup with optional filtering, compression, logging, desktop links for jobs, orphan file removal, and selection of destination based on volume label instead of drive letter.  Screenshot

/windows | Feb 15, 2010

Delete multiple pages in a DjVu document with djvm #

djvm (a command line tool bundled with DjVuLibre), does not accept multiple pages or page ranges for the delete argument. Here are a few workarounds for deleting all pages in a range, even pages, or odd pages: Many thanks to DutchDaemon and s0xxx for their answers in this thread, and to SilentGhost for this answer on Stack Overflow.

/nix | Feb 15, 2010

Mojikyo Character Map #

The Mojikyo Character Map is a freeware application (bundled with the necessary TrueType fonts) that provides easy access to =~ 110,000 mainly Chinese characters:

Mojikyo Character Map

I stumbled onto the Mojikyo Character Map in a blog post by Jonathon Delacour from 2003 that alluded to 34 files totaling 52MB. Since the Mojikyo website and Stanford mirror no longer offer the files, I started poking around the Internet Archive cache of the Stanford mirror, which has all of the necessary files but one (found elsewhere - see below):
  1. Downloaded the compressed application, MOCM400.EXE, located in the CMAP subdirectory
  2. Batch downloaded (via Orbit Downloader) the 33 compressed fonts found in WINTTF
  3. MOJIKM5E.EXE was the only file not available, but I found a copy on the Chinese GNU/Linux Extensions website (which turns out to host all of the necessary files).
  4. I now had 34 files totaling =~ 52MB:
     MOCM400.EXE   1,572,268  04/02/2002  09:46 AM  3180072780
     MOJIKM01.EXE  2,250,455  07/19/2002  01:49 PM  2318009370
     MOJIKM02.EXE  2,348,019  07/19/2002  01:49 PM  3277625571
     MOJIKM03.EXE  2,323,762  07/19/2002  01:49 PM  4265428117
     MOJIKM04.EXE  2,371,299  07/19/2002  01:49 PM  3923032980
     MOJIKM05.EXE  2,384,680  07/19/2002  01:49 PM  2234262401
     MOJIKM06.EXE  2,482,608  07/19/2002  01:49 PM  3820311365
     MOJIKM07.EXE  2,316,520  07/19/2002  01:49 PM  226176670
     MOJIKM08.EXE  2,531,450  07/19/2002  01:50 PM  2244807325
     MOJIKM09.EXE  2,775,561  07/19/2002  01:50 PM  2001805475
     MOJIKM0A.EXE  2,063,528  07/19/2002  01:50 PM  3828274974
     MOJIKM0B.EXE  2,074,818  07/19/2002  01:50 PM  178146578
     MOJIKM0C.EXE  1,770,878  07/19/2002  01:50 PM  2974162806
     MOJIKM0D.EXE  1,986,658  07/19/2002  01:50 PM  2084838314
     MOJIKM0E.EXE  2,153,982  07/19/2002  01:50 PM  3849468493
     MOJIKM0F.EXE  2,715,471  07/19/2002  01:50 PM  2125653168
     MOJIKM10.EXE  1,603,651  07/19/2002  01:50 PM  1683753854
     MOJIKM11.EXE    376,114  04/02/2002  10:06 AM  3419606807
     MOJIKM12.EXE  1,675,983  04/02/2002  10:06 AM  3778053224
     MOJIKM13.EXE    683,293  04/02/2002  10:06 AM  2249996894
     MOJIKM14.EXE    292,419  04/02/2002  10:06 AM  1234093088
     MOJIKM15.EXE    253,070  04/02/2002  10:06 AM  2318227551
     MOJIKM55.EXE    194,289  04/02/2002  10:06 AM  399366025
     MOJIKM56.EXE    810,418  04/02/2002  10:06 AM  1533657018
     MOJIKM57.EXE  1,706,312  04/02/2002  10:06 AM  2802977783
     MOJIKM58.EXE  1,686,681  04/02/2002  10:06 AM  255553839
     MOJIKM59.EXE  1,226,551  04/02/2002  10:06 AM  221434629
     MOJIKM5A.EXE    473,245  04/02/2002  10:06 AM  2926850122
     MOJIKM5B.EXE  1,603,179  04/02/2002  10:06 AM  3163789964
     MOJIKM5C.EXE  1,648,109  04/02/2002  10:06 AM  137386859
     MOJIKM5D.EXE    978,290  04/02/2002  10:06 AM  4273653570
     MOJIKM5E.EXE    358,966  04/02/2002  10:06 AM  1539610454
     MOJIKM66.EXE  2,197,871  04/02/2002  10:06 AM  4218732064
     MOJIKM67.EXE    591,716  04/02/2002  10:06 AM  251927137
  1. Batch uncompressed with 7-Zip and removed unnecessary files, leaving 34 totaling =~ 94.4MB:
     MOCHRMAP.exe    532,480  02/08/2002  04:00 AM  458313828
     Mojikm01.TTF  3,815,992  06/20/2002  12:00 AM  2387683204
     Mojikm02.TTF  4,112,612  06/20/2002  12:00 AM  2117129841
     Mojikm03.TTF  4,127,344  06/20/2002  12:00 AM  1634353783
     Mojikm04.TTF  4,234,524  06/20/2002  12:00 AM  4286314746
     Mojikm05.TTF  4,426,808  06/20/2002  12:00 AM  845998767
     Mojikm06.TTF  4,430,780  06/20/2002  12:00 AM  4162377448
     Mojikm07.TTF  4,431,964  06/20/2002  12:00 AM  3159972071
     Mojikm08.TTF  4,999,792  06/20/2002  12:00 AM  4102742411
     Mojikm09.TTF  5,415,780  06/20/2002  12:00 AM  3747794663
     Mojikm0A.TTF  3,707,676  06/20/2002  12:00 AM  3723578800
     Mojikm0B.TTF  4,177,592  06/20/2002  12:00 AM  2228260563
     Mojikm0C.TTF  4,080,532  06/20/2002  12:00 AM  3033408358
     Mojikm0D.TTF  3,519,976  06/20/2002  12:00 AM  2079183546
     Mojikm0E.TTF  3,496,484  06/20/2002  12:00 AM  2932782927
     Mojikm0F.TTF  4,482,840  06/20/2002  12:00 AM  3892262837
     Mojikm10.TTF  2,551,176  06/20/2002  12:00 AM  875814641
     Mojikm11.TTF    694,136  10/24/2001  02:01 AM  931566581
     Mojikm12.TTF  2,482,876  10/24/2001  02:02 AM  1225778174
     Mojikm13.TTF  1,163,308  10/24/2001  02:02 AM  3841779345
     Mojikm14.TTF    591,264  10/24/2001  02:01 AM  2458869305
     Mojikm15.TTF    530,880  10/24/2001  02:02 AM  2522906767
     Mojikm55.TTF    410,636  02/14/2002  02:10 AM  3210542770
     Mojikm56.TTF  1,474,892  10/24/2001  02:01 AM  3142615718
     Mojikm57.TTF  3,426,016  10/24/2001  02:01 AM  2234440663
     Mojikm58.TTF  3,520,136  10/24/2001  02:01 AM  3854002523
     Mojikm59.TTF  2,361,240  10/24/2001  02:01 AM  3428091862
     Mojikm5A.TTF    981,012  10/24/2001  02:01 AM  1099258963
     Mojikm5B.TTF  3,301,032  10/24/2001  02:01 AM  3743177567
     Mojikm5C.TTF  3,705,536  10/24/2001  02:01 AM  3218894549
     Mojikm5D.TTF  1,940,452  10/24/2001  02:01 AM  4040218154
     Mojikm5E.TTF    752,000  10/24/2001  02:01 AM  1420675346
     Mojikm66.TTF  3,951,240  02/14/2002  02:11 AM  4149409031
     Mojikm67.TTF  1,141,180  02/14/2002  02:11 AM  2422714417
  1. Copied the TTF files to %WinDir%\Fonts
  2. Launched MOCHRMAP.exe
Two final notes: The University of Virginia website hosts what appears to be a slightly older version of the application and associated fonts. Also, OS X's Character Palette provides similar functionality as the Mojikyo Character Map, though it does not contain all of the same characters by default.

/windows | Feb 13, 2010

Using a local printer with Remote Desktop #

Setup remote printing over RDP:

1. Install printer driver on remote machine
Unless the remote machine already has a driver for the local printer, you will need to install it:
  Windows XP: Printers and Faxes > File > Server Properties > Drivers > Add
  Windows 7: Administrative Tools > Print Management > Print Servers > computer name (local) > right click Drivers > Add Driver...
Once the driver has been added, log off the remote machine

2. Enable printer access in client PC's Remote Desktop Connection
Remote Desktop Connection > Options > Local Resources > check "Printers" under Local Devices > Connect. The local printer becomes the default for programs running on the remote desktop.

3. Notes

/windows | Feb 10, 2010

Restore tab autocompletion in cmd.exe #

Head to
  HKEY_CURRENT_USER\Software\Microsoft\Command Processor
  HKEY_LOCAL_MACHINE\Software\Microsoft\Command Processor
(if the HKCU value is set, it overrides the HKLM value) and set the CompletionChar value to 9 (which is horizontal tab in ASCII).

/windows | Feb 07, 2010

Determine Windows XP Service Pack level from registry #

  C:\>reg query "HKLM\Software\Microsoft\Windows NT\CurrentVersion" /v CSDVersion

Besides System Properties (sysdm.cpl), here are two other standard GUI apps that return the Service Pack level: winver.exe and winmsd.exe. Just launch from the Run dialog or a command prompt.

/windows | Feb 06, 2010

Outlook Express opens PDF attachments with wrong version of Adobe Acrobat #

While double clicking PDFs from within Windows Explorer opened them in Adobe Acrobat Reader 9 (the default PDF handler), double clicking PDF attachments from within Outlook Express opened them in Adobe Acrobat 6. This issue was resolved by changing the values of


to the path of the current version of Adobe Acrobat Reader:

  "C:\Program Files\Adobe\Reader 9.0 Reader\AcroRd32.exe" "%1"

(Thanks to wimvh for digging up this fix)

/windows | Feb 06, 2010

Linux Mint 8 (Helena) - Add Firefox shortcut / launcher to Desktop #

This should really be simpler...
  1. Right click on Desktop and select "Create Launcher"
  2. Click the springboard icon at top left
  3. Enter the following path: /usr/share/pixmaps and then double click the Firefox icon
  4. Enter the values below:
  5. Click OK

/nix | Feb 06, 2010

Keynes is whack, yo. #

/misc | Jan 30, 2010

Batch convert HTML to PDF in OS X #

HTML files can be batch converted to PDF under OS X using a for loop and the virtually undocumented convert command (not the one from ImageMagick):

  for file in *.html;do /System/Library/Printers/Libraries/convert -f $file -o $(basename -s ".html" "$file").pdf;done

/mac | Jan 24, 2010

HTML Tidy: Batch processing files #

HTML Tidy does not natively support wildcards in filenames (e.g., *.html), but batch processing in bash is possible with a simple for loop:
  for f in *.html;do tidy -m -i $f;done
-m = modify original input files
-i = indent element content

(For a complete list of arguments, see the man page.)

/nix | Jan 24, 2010

Dell MFP 3115cn network scanning error: Network Not Ready 016-790 #

When connected to a network, the Dell 3115cn can send scans to a SMB share or FTP server. The setup process could be clearer, as evidenced by posts like these: We resolved a "Network Not Ready 016-790" error message from the panel: SCAN > Scan Defaults > Network Port > changed interface from the unused wireless to wired. This can also be done from the web interface: Printer Settings > Printer Settings > Scan Defaults > Network Port. While you're there, here are the SMB settings that worked for us:
  1. Address Book > Server Address > Create
  2. Name: enter a short, friendly name
  3. Server Type: SMB
  4. Server Address: use static IP of SMB server
  5. Server Port Number: 139
  6. Login Name and Login Password: account which exists on SMB server and has access to the shared directory
  7. Share Name: name of shared directory
  8. Server Path: blank

/misc | Jan 23, 2010

Two newly-listed Palm OS apps #

/palm | Jan 23, 2010

Hide desktop icons #

Unclutter your OS X desktop instantly (for taking screenshots, giving presentations, faking fastidiousness, etc.):

/mac | Jan 17, 2010

Quick contact sheets in OS X #

Macworld explains how to quickly create an image contact sheet in Leopard:
Open folder in Finder > Command-A > Command-Option-Y > click Index Sheet button > Command-Shift-3.

However, since images which scroll off the screen are not captured, this little app may be a better option:
ContactSheetMaker 1.0.1 [53k] {S} Simple, standalone application to create contact sheet image files from folders of images.  Screenshot

/mac | Jan 13, 2010

Google does the right thing #

A new approach to China: "We have decided we are no longer willing to continue censoring our results on Google.cn, and so over the next few weeks we will be discussing with the Chinese government the basis on which we could operate an unfiltered search engine within the law, if at all. We recognize that this may well mean having to shut down Google.cn, and potentially our offices in China."

UPDATE 1: Apparently this was all talk, no walk: Google co-founder: Maybe we'll stay in China after all - A month after storming the moral high-ground over China's hacking activities, Sergey Brin has declared the firm is happy to get off its high horse and kick its heels in the country a little longer.

UPDATE 2: Finally some real action: So earlier today we stopped censoring our search services - Google Search, Google News, and Google Images - on Google.cn. Users visiting Google.cn are now being redirected to Google.com.hk, where we are offering uncensored search in simplified Chinese, specifically designed for users in mainland China and delivered via our servers in Hong Kong.

/misc | Jan 13, 2010

Cannot uninstall ZoneAlarm: "missing a necessary root certificate" #

Simply set the system clock back to 2007 or 2008 and relaunch uninstaller to bypass the root certificate error message.

/windows | Jan 12, 2010

Newly added icon editor for OS X #

Iconographer X 2.5 [891k] Full-featured icon editor. Formerly shareware, now free (registration name = "Iconographer is now free", code = "HG066414").  Screenshot

/mac | Jan 04, 2010

Detect the character encoding of a file #

The aforementioned Perl module Unicode::Japanese includes ujguess, which attempts to detect the character encoding of a given file. The Unix program file is often suggested on forums and the like for this purpose, but it only returns the file type, not the encoding. Here's an illustration of the difference, using a Shift JIS-encoded file:
$ file foo
foo: UTF-8 Unicode text, with no line terminators

$ ujguess foo
and an EUC-encoded one:
$ file bar
bar: ISO-8859 text, with CRLF line terminators

$ ujguess bar

/nix | Jan 03, 2010

Create a universal Windows 7 install disc #

Setting aside the absurdity of offering a confusing array of editions, Microsoft made technicians' jobs slightly easier by including all editions of Vista (Home Premium, Ultimate, etc) on a single 32 or 64 bit disc (though both versions can apparently be combined with a bit of effort). This allowed techs to carry one or two discs at most and still be able to install whatever edition was required. Windows 7 changes all that; while the various and sundry editions are all tantalizingly included on the disc, a tiny configuration file (ei.cfg) forces a specific edition to be installed. Thankfully, Kai Liu has created win7utils to circumvent this irritation: (via Mike Mills who found it on Lifehacker which credited Technibble which attributed the link to a forum member named AtYourService - whew!)

/windows | Jan 03, 2010

Convert numbers and spaces from full-width (double-byte) to half-width (single-byte) #

Within filenames (using Bash, Perl, and Unicode-Japanese-0.47):
  1. $sudo perl -MCPAN -e shell
  2. cpan> install Unicode::Japanese
  3. Save the following script (adapted largely from togdon's example) as full2half.sh and make it executable (chmod u+x):
    for file in *;do
    newfile=$(echo $file | perl -MUnicode::Japanese -e'print Unicode::Japanese->new(<>)->z2h->get;')
    test "$file" != "$newfile" && mv "$file" "$newfile"
  4. cd to desired directory and run script:
    $ /path/to/full2half.sh
    Convert full-width numbers and spaces to half-width
Within file contents (using OpenOffice.org):
  1. Tools > Options... (in OS X it's OpenOffice.org > Preferences...)
  2. Language Settings > Languages > check "Enabled for Asian languages" > OK
  3. Highlight text > Format > Change Case > Full-width or Half-width

/nix | Jan 02, 2010

Blue screen when booting into Safe Mode on malware-infected machine #

may be resolved by restoring the SafeBoot registry keys. You may need to create the SafeBoot registry key with special permissions protecting it from deletion. Tags: BSOD, Safe Mode, STOP 0x0000007B

/windows | Jan 01, 2010

Subscribe or visit the archives