Day-to-day thoughts, technical information, a random grab-bag of thoughts, discoveries, and interesting little tidbits.

Tuesday, December 15, 2009

Sounding Off

I finally got around to learning how to use the JACK Audio Connection Kit, which, as far as I'm concerned, is easily a contender against the big names like Pro Audio, Cubase, or any sound program by Adobe.  Sadly, Audacity doesn't cooperate with the virtual patch panel, but the Ardour DAW seems to fill in the gaps quite nicely.  Combine this with Jack Rack (a tool to integrate various sound filters into the patch panel) and Jamin (a audio mastering interface), and there's a nigh-professional sound editing system at my fingertips.  And if I need to do some post-processing, well, Audacity will fit the bill quite nicely.

I could use this to produce some interesting things; I have had some recent interest in performing audio theatre podcasts, perhaps do some voice-over work in the longer-term future... who knows?

Jack Audio Connection Kit
Audacity Sound Editor
Ardour Digital Audio Workstation
Jack Rack Audio Effects Toolkit
Jamin Audio Mastering Interface

Sunday, October 4, 2009

Cleaning up Scanned Documents with the GIMP

Recently, I've decided that I don't really want to keep a lot of paper documents lying around, such as bills and receipts, but I also don't want to lose the information they contain.  So, I decided to scan them into my computer, and store them as images, eventually to be converted into PDF format.

After scanning in the first document, I came across the problem that no doubt a lot of you encounter; the scanned document is sort of off-kilter, no matter how determinedly I aligned the paper on the glass.  Additionally, the document's background seemed to be inconsistently shaded at best.  Of course, there's also the ever-present black border where the paper did not cover the glass.  Obviously, this would not do.

After an hour attempt, I started to put together a method of cleaning up documents to the point that they would be cleanly replicated on the image file, to the point that signatures could even look realistic.  This article describes just how this happens, and how it can be corrected.  For simplicity's sake, I will describe this using the free Photoshop workalike known as the GNU Image Manipulation Program (or Gimp, as it's usually shortened to).

Common Issues and their Causes

The following image shows the basic layout of a scanner and the document placed in it.

The blue box at the right is the scanner's lamp and imaging system; it goes over the glass at a consistent rate, and constantly takes snapshots of its location as it goes.  Under perfect conditions, where the paper is perfectly aligned and flat, where the lamp is the only source of light there is, and where the lamp/sensor itself is perfectly aligned, this can result in a perfectly clean scan with no artifacts or strange bending.

However, such perfect scanners generally cost in the megabucks, so the typical scanner is generally "close enough," in that the document is clean, clear, and not too badly warped, and is mostly legible.  The developers of such scanners expect the cleanup to be done in software, so the tolerances can be a bit more off than the more expensive devices.

You'll notice with the above pictures that the blue lamp/sensor device isn't completely aligned with the paper.  It will still scan straight, but it results in one of the more common faults of scanned documents: shear.  This means that one side of the scan will be captured earlier than the other side, causing the whole document to look crooked, as the above illustration shows.  What makes this a bit more annoying is that rotation alone won't fix the problem, as it will simply rotate the text as well as the lines and borders.

Another problem that often pops up is the strange "fog" of color that warps over the image, usually marbled based on imperfections in the paper; the lamp puts out a steady light that is reflected off the paper and returned to the sensor on the assembly.

 As you can see, the first example shows a perfect scan, if the paper is perfectly flat and even.  The second example is a bit more realistic; paper consists of millions of miniature bumps, and is usually something less than perfectly flat, especially if it was handled in some way, and definitely if it was recently folded or crumpled.  In this case, those areas that are further from the glass (and lamp) become slightly darker, and the texture of the paper itself can affect the darkness somewhat.  This is the source of the colored (or gray) "fog" that can cover a good portion of the paper.

Example Document Used

 The following document will be used for the examples; it was scanned using the green channel at 300dpi with an 8-bit grayscale.  It was originally part of a larger, but mostly-obsolete document I had stored in a 3-ring binder for a while (for those curious, it was from the "Encrypted Root Filesystem HOWTO" website).
The reasons for my scanning choices are as follows:
  • 300dpi is the typical resolution for a mid-grade laser printout of a document.
  • The green channel generally has the least defects (less splotchy fog, and more smooth, faint fog)
  • The 8-bit grayscale will cover approximately 256 shades of gray (makes an excellent starting point toward clearing the fog).

Correcting Document Shear

The first issue that will need to be resolved is the straightening of the document.  This is not very hard to accomplish, but it does require some patience and practice... and a little help with the guide and shear tools in the Gimp.

The first step involves a calculation: You need to determine how wide your total scan is, and then place a vertical guide at the exact (or bordering) halfway point.  Then, you need to locate a line long enough on the document to compare ends, and align a horizontal guide exactly where the vertical guide and the document's line meet:

In the case of my example, my flatbed scanner has a maximum width (at 300dpi) of 2563 pixels, so 1281 would be an appropriate location for the cross.  Also, notice the dotted blue line going across.  On the far left and far right of that border, the blue line is above and below it, respectively.

Now that the guides have been placed, we can now use the shear tool in the Gimp.  In the toolbox on the right, look for the shear tool:

Up will pop the shear dialog box, which contains two options and four buttons:

 When you make changes to the values in the magnitude options, the displayed image will automatically update as a preview.  The easiest way to ensure you don't overshoot the mark is to use the up and down arrows to the right of the entry fields to take the changes one step at a time.  In this case, we want to change the "Shear magnitude Y."

The above picture shows the before; here you see the shear box with both 0 entries, and the image below with the two guides.  Notice that the horizontal guide shows just how bad the shear is on the document.  In the following image, however, all of this is corrected:

Here, you can see that the lines are now matched up, and since the guide lines are always perfectly horizontal and vertical, you know that the document has been corrected.  Something to keep in mind, however: when you try this, you'll notice that while you're doing the shearing, the lines will become jagged and uneven.  Once the process is completed, the jagged lines will be corrected, and everything will be smooth and straight.

Correcting "Scanner Fog"

Now, you'll notice that the image does have a hazy gray color over most of the surface.  Most of the time, the haze is paler than anything else on the page, which means that cleaning it up is just a levels tool away.

The levels tool is designed to change how light the lights are, and how dark the darks are.  You can activate the levels tool by going to the "Colors" menu and clicking on "Levels..."  You will then see the dialog box shown below:

Most of what you see on this dialog box is unimportant for our purposes; all we're interested in is the histogram and the input level controls.  The histogram is the big white box above, with the black "mountain" shown in it.  Beneath it, you see three triangles under a gradient.  We'll call those triangles the blackpoint, greypoint, and the whitepoint, as they mark where the black, middle gray, and white are in the image.

The histogram is essential to the task of cleaning the fog.  You'll notice the mountain showing up at the far right of the dialog.  That mountain happens to be just above the very light gray part of the grayscale.  This means that each of those shades of gray have a lot of pixels of that brightness.  At this point, you probably realize that the mountain describes the pixels in the scanner fog, which is usually in that brightness range, as compared with the actual text, which is considerably darker.

Underneath the histogram, we have the grayscale with three triangles.  Those triangles generally adjust where the black color starts lightening up, and where the white color begins to darken.  To fix the problem with the scanner fog, we need to move the white arrow to the point just left of the mountain, which will turn all the pixels of scanner fog pure white, essentially losing those pixels as data (don't click OK just yet):

But, now we have a new problem: the text in the document has become quite faint!  Well, that can be corrected as well.  Notice the low, long hill on the left side of the histogram?  Those are the pixels that contain the content, much of which was black on paper.  We just need to move the black slider to the right so that hill is above the black region at the bottom:

As you can see, the document has been darkened again, but the fog is still gone.  As you can see, the "fog mountain" in the histogram is now above the pure white region, making all "fog" pixels white, while the "content hill" is now completely above the pure black region, meaning that much of the content is now back to black.  For the most part, the corrections are mostly done.  To remove the punch-holes on the left-hand side, a quick swipe of white paint will fix it in no time.

Now, you'll notice that the borders are a little off, and, of course, the black border is still there.  Cropping is the word for this occasion; a tool that will delete everything outside of a rectangular region you select.  Simply open the tool, adjust the borders to match where you want the page cut, and voila.  To begin, you need to select the crop tool from the toolbox:

Then click and drag from one corner of your desired selection to the other, using the boxes along the edges, and over the corners to resize the selection to crop. It is recommended that you also make use of the zoom function if you want to crop only what is necessary to clear away the outer edges:

To complete the cropping process, you simply need to click on the inside of the crop selection, and it is done.

You might notice that the gray regions are not all that even.  This can have multiple reasons, but the main reason is often that the printer that produced the image was not perfect, either, putting more ink/toner into some dots/regions than others.  In this document's case, it was getting near the time when fresh toner would be needed.  Even though these techniques cleaned up the document quite nicely, remember that nothing changes the fact that unless you actively change the document yourself, you can at best get on the screen exactly what was on the paper you scanned.

Well, enjoy this technique, and good luck!

Sunday, June 28, 2009

Making Markets in EVE Online

In EVE Online, trading has a simple concept: Buy low and sell high. If you're lucky, you can find an excellent trade route to make your fortune. However, many traders do not seem to understand another simple concept: You make your own luck.

Instead of trying to look for places with low and high prices, why not place orders far and wide, and then sell the purchased goods in a high-trade location?

I have found that if you purchase goods at 40-60% of their default prices, and then sell them at 60-80% of their default prices, you can actually make a profit from every single sale. In order for this to work, you need to span the purchase orders as far as you can safely travel, then travel out to pick up the goods, and bring them back up to a high-population area to sell, such as Jita. Remember to use the star charts to avoid potential ambush points; always avoid locations with recent podkilling.

Essentially, when trading, you can either wait for the market to give you a profit, or mold the market to give you your profit.

Another page does a much better job of explaining methods to avoid becoming subatomic particles during your collection runs:

Thursday, February 26, 2009

Self-Made vs. Pre-Made Computers

I have broken down and purchased something I never thought I would... a branded computer.

Not since the Commodore 64 have I ever purchased a pre-assembled computer, as I've had enough bad experiences with the ones I've dealt with on a more professional level. IBM, Gateway, Dell, Hewlett-Packard, Compaq, and especially E-Machines and Packard-Bell (ugh!), it didn't matter. I would (and generally still do) avoid pre-assembled computers like the plague.

"Why is this?" I can hear you ask.

If I didn't build a computer, then there would always be a chance that the parts would be 'custom-made' to 'interact with each other,' and any replacements I make would potentially result in hard-to-track problems as parts used to dealing with one another suddenly no longer have the 'one another' to interact with.

This is doubly so when part of the pre-installed software interacts with these specialized parts for the sake of 'security' or 'stability.' Essentially, the idea seems to be focused on 'vendor lock-in,' because anything not vendor-based is a 'security risk.' Packard Bell was particularly obsessive in this area. And then, there is the shovelware; you know, all those nigh-useless, house-branded programs, registration, and advertisement icons that are preinstalled with the computer when you buy it? On top of this, specialized software can have bad results if uninstalled. Shovelware doesn't. And the vendor does not make it easy to tell which is which.

So, with all this dislike of mass-produced computers, you are wondering why the breakdown. Where the premade/selfmade dichotomy breaks down is in one area: laptops.

A laptop is designed to be self-contained; very few parts are replaceable in a laptop; you need to be comfortable with the display and the keyboard. And since they are integrated with the case, which is integrated with the motherboard... well... there's not much left to choose from. You can't change the keyboard layout, can't choose a bigger display or a quieter case, and removing the inbuilt pointing device would be more work than it's worth.

In any event, I've bitten the bullet, and purchased one of the cheaper laptops, specifically an Acer 5515-5831 laptop. I've completely wiped Windows (and the hidden restore partition) from the computer, and am installing Gentoo now. We'll see how it works.