Wednesday, January 25, 2012

Text File Processing With Bash

This took me way too long to figure out, and I'll never remember how to do it, so I document it here for all time. I had this list of files I needed to download using wget or curl or whatever. The trick was that each file had a unique URL and required a unique output filename. So I had this long-ish text file which looked like:

127_368_2.pdf http://localhost:52225/Cache/Embedded/127/368/2.pdf
127_368_3.pdf http://localhost:52225/Cache/Embedded/127/368/3.pdf
923_7_1.pdf http://localhost:52225/Cache/Embedded/923/7/1.pdf

where the first string was the output name and the second was it's source URL. After bouncing around scripting forums for way, way too long, I came up with the following:

cat files.txt | while read x y; do curl -b authcookie.txt -o $x $y; done

What took me so long to figure out was the parsing of each line of the file, which "while" takes care of on it's own when multiple arguments are provided. Neat.

SSH Tunneling for Fun and Profit

When you finally, finally understand how SSH tunneling works, it opens up a whole universe of possibilities...

Dynamic DNS

If you don't have a static IP address, you'll need to get yourself set up with a dynamic DNS provider so that you can find your computer no matter what IP address your ISP has given you. There are plenty of service providers to pick from. I like DynDNS because it's free and my provided domain name can be updated from my router (they also provide installable daemons for any flavor of OS).

Tunnel through NAT (with static DHCP leases)

You've got a few options for trying to tunnel through to a SSH server behind your NAT/firewall. You can set up your router to forward a certain port to your SSH server, or if your router itself has an SSH server, the ssh tunneling syntax can do the forwarding for you:

ssh -L 9999:targetserver:1234 sshuser@sshserver

This command will tunnel port 9999 on the local machine to port 1234 on the target machine through the SSH server. Keep in mind that the data flow between the SSH server and target server will not be encrypted. This issue is addressed later.

Stream Ampache (or any web application)

Simple enough--build a tunnel to the web server's http port.

ssh -L 8888:webserver:80 remoteuser@sshserver

Then point your web browser to http://localhost:8888.

VNC over SSH

Here's a handy scenario for tunneling a VNC server to local port.

ssh -L 5900:vncserver:5900 remoteuser@sshserver

Now just point your VNC client application to localhost:5900 and bam! you're connected. With this tunnel, traffic between sshserver and vncserver is exposed to the network, and since VNC is not a secure protocol by default, this could be an issue. The next section demonstrates a potential solution.

Tunnel Within a Tunnel (We Have to Go Deeper) (via1, via2)

Say you need to SSH tunnel from your localhost to host1, and then from host1 to host2. Provided that host1 and host2 both have SSH servers, you can join two tunnels in the following way:

ssh -L 9999:host2:22 host1user@host1
ssh -L 9998:localhost:5900 -p 9999 host2user@localhost

The first tunnel connects the SSH server on host2 port 22 to localhost port 9999 (using host1's SSH server). The second tunnel connects the service at port 5900 on host2 to the localhost port 9998 (through host2's SSH server available now as a result of the first tunnel).

So when whatever client application is pointed to the localhost port 9998, it's really tunneling to host2 port 5900 through host2's SSH server... where the connection to host2's SSH server is actually through another tunnel (from localhost port 9999 to host2 port 22 over host1's SSH server). And there you have it: a tunnel within a tunnel. My head hurts.

Stream iTunes (via1, via2)

This gets a little trickier because of the way iTunes advertises itself on the network. We'll need to use another application to assist with that bit, but the tunneling is the same. iTunes streams over port 3869, so:

ssh -L 9999:itunesserver:3689 sshuser@sshserver

Then, with the tunnel up and running,

dns-sd -P "Home iTunes" _daap._tcp local 3689 localhost.local. 127.0.0.1 "Arbitrary text record"

Package that up into a script, and you're good to go.

Friday, January 20, 2012

Pipe Char


Monday, January 16, 2012

jQuery Pan & Zoom for Images and Maps

This is a nice little gadget for large image viewing:
http://wayfarerweb.com/jquery/plugins/mapbox/

I'm thinking of adding a few improvements:

  1. "Smoothing out" transitions between zoom levels
  2. Support for HTML5 gestures allowing pinch-to-zoom on mobile devices
  3. Kinetic scrolling/panning
Updates to come...

Google Voice Call Screening and the iPhone

If you have Call Screening active, and you answer a call from your Google Voice forwarding number, you must press "1" in order to answer the call (or otherwise navigate a touch-tone menu system). Don't know about you, but I find this to be a minor but annoying inconvenience.

You can turn off Call Screening, but there are two sides to every Schwarz, and while the call will now connect with no confirmation required, your phone's voicemail will engage before Google voicemail--thus losing all those neat Google voicemail advantages.

But have no fear; there is a solution. Just extend your phone's voicemail timeout to be of a longer duration than the Google voicemail timeout, thereby ensuring an unanswered call is always sent to Google Voice's voicemail system before it reaches your provider's voicemail system.

In other words:
http://support.google.com/voice/bin/answer.py?hl=en&topic=19490&answer=115110

Now I can't speak for other carriers and phone models, but in order to set the voicemail timeout for an AT&T iPhone 4, the following instructions worked fine for me (source):

  1. Dial "*#61*".
  2. Copy down the number in the text "Voice Call Forwarding When Unanswered Forwards to +1XXXXXXXXXX Enabled"
  3. Dial *61*+1XXXXXXXXXX*11*tt# where +1XXXXXXXXXX is the number you copied in the previous step, and "tt" is the number of seconds (from 5 to 30 in increments of 5) for the voicemail timeout duration you wish.
As the Google Voice timeout is 25 seconds, I recommend setting your carrier timeout to 30 seconds.

Update: There's this, too.

Saturday, January 7, 2012

Entity Framework, Unit of Work, and Shared Context

Those familiar with Entity Framework, MVC, and the repository pattern are also familiar with the following run-time error:
The relationship between the two objects cannot be defined because they are attached to different ObjectContext objects.
It occurs when one attempts to associate an entity object from one data context to an entity object from another data context. With the repository pattern, there is usually one data context per repository. So if, for example, one were to query for a user

user = usersRepository.Find(userId);

and a role

role = rolesRepository.Find(roleId);

and then attempt to associate the two

user.Roles.Add(role);

one would get the aforementioned error when executing the line

usersRepository.Save()

where the Save() method wraps the SaveChanges() method of the data context.

This post describes the situation very well. It also describes some potential solutions, one of which is to share the data context across all repositories for the life of the web request. And one way to do that is to use DI--in my case, StructureMap.

Each of my repositories ultimately derives from an abstract repository base class.

public abstract class RepositoryBase
{
 protected DbContainer db;

 public RepositoryBase()
 {
  db = ObjectFactory.GetInstance<DbContainer>();
 }
}

And over in the DI configuration:

ObjectFactory.Initialize(x =>
{
 x.For<DbContainer>().HttpContextScoped().Use(() => new DbContainer());
 // ...
});

And there we have it, folks. On a cautionary note, because all repositories share the same context, a call to a repository's Save() method will persist changes to any changed entity, not just the ones from the saving repository. This can result in unexpected behavior, so, you know, be aware.

Wednesday, January 4, 2012

Using ManualResetEvent to Wait

This one time, at band camp, I was using a PDF library from third-party to generate PNG thumbnails of the pages. The library was set up in such a way that thumbnail generation was forked to a separate thread with only an event handler to notify when rendering was complete.

Since this code was going to serve the thumbnail file from a web server, I needed to file to be ready before delivering the web response. I had to wait until the event fired--a sort of "re-synchronization" or "de-asynchronization", if you will.

Enter the ManualResetEvent. With some blogger help, I was able to put together the following:

using System;
using System.Drawing;
using System.Drawing.Imaging;
using System.Threading;

// http://code.google.com/p/pdfviewer-win32
using PDFLibNet;

namespace InSite.Portal.Helpers
{
    public class PdfHelper
    {
        PDFWrapper _pdf;

        public PdfHelper(string sourceFilePath)
        {
            _pdf = new PDFWrapper();
            _pdf.LoadPDF(sourceFilePath);
        }
       
        public void SavePageAsImage(string destFilePath, int page)
        {
            SavePageAsImage(destFilePath, page, ImageFormat.Png);
        }
        
        public void SavePageAsImage(string destFilePath, int page, ImageFormat format)
        {
            ManualResetEvent _mre = new ManualResetEvent(false);
            var renderPageThumbnailFinished = new RenderNotifyFinishedHandler((i, s) => { _mre.Set(); });

            var pg = _pdf.Pages[page];
            pg.RenderThumbnailFinished += renderPageThumbnailFinished;
            var h = Convert.ToInt32(pg.Height); // get height
            var w = Convert.ToInt32(pg.Width);  // get width
            var bmp = pg.LoadThumbnail(w, h);   // call thumbnail render
            _mre.WaitOne(30 * 1000, true);      // wait until render complete (30 sec timeout)
            bmp.Save(destFilePath, format);     // save completed image
            pg.RenderThumbnailFinished -= renderPageThumbnailFinished;
        }

        public int PageCount
        {
            get
            {
                return _pdf.PageCount;
            }
        }
    }
}

Crude? Elegant? Either way, it gets the job done.