UnxUtils + cURL

For when you need to do some basic unix-type scripting but don't want the overhead of installing cygwin:

• UnxUtils: Bins that will run on Windows with no external dependencies.
• cURL: Because wget is for little girls.

Text File Processing With Bash

This took me way too long to figure out, and I'll never remember how to do it, so I document it here for all time. I had this list of files I needed to download using wget or curl or whatever. The trick was that each file had a unique URL and required a unique output filename. So I had this long-ish text file which looked like:

127_368_2.pdf http://localhost:52225/Cache/Embedded/127/368/2.pdf
127_368_3.pdf http://localhost:52225/Cache/Embedded/127/368/3.pdf
923_7_1.pdf http://localhost:52225/Cache/Embedded/923/7/1.pdf

where the first string was the output name and the second was it's source URL. After bouncing around scripting forums for way, way too long, I came up with the following:

cat files.txt | while read x y; do curl -b authcookie.txt -o $x $y; done

What took me so long to figure out was the parsing of each line of the file, which "while" takes care of on it's own when multiple arguments are provided. Neat.

SSH Tunneling for Fun and Profit

When you finally, finally understand how SSH tunneling works, it opens up a whole universe of possibilities...

Dynamic DNS

If you don't have a static IP address, you'll need to get yourself set up with a dynamic DNS provider so that you can find your computer no matter what IP address your ISP has given you. There are plenty of service providers to pick from. I like DynDNS because it's free and my provided domain name can be updated from my router (they also provide installable daemons for any flavor of OS).

Tunnel through NAT (with static DHCP leases)

You've got a few options for trying to tunnel through to a SSH server behind your NAT/firewall. You can set up your router to forward a certain port to your SSH server, or if your router itself has an SSH server, the ssh tunneling syntax can do the forwarding for you:

ssh -L 9999:targetserver:1234 sshuser@sshserver

This command will tunnel port 9999 on the local machine to port 1234 on the target machine through the SSH server. Keep in mind that the data flow between the SSH server and target server will not be encrypted. This issue is addressed later.

Stream Ampache (or any web application)

Simple enough--build a tunnel to the web server's http port.

ssh -L 8888:webserver:80 remoteuser@sshserver

Then point your web browser to http://localhost:8888.

VNC over SSH

Here's a handy scenario for tunneling a VNC server to local port.

ssh -L 5900:vncserver:5900 remoteuser@sshserver

Now just point your VNC client application to localhost:5900 and bam! you're connected. With this tunnel, traffic between sshserver and vncserver is exposed to the network, and since VNC is not a secure protocol by default, this could be an issue. The next section demonstrates a potential solution.

Tunnel Within a Tunnel (We Have to Go Deeper) (via1, via2)

Say you need to SSH tunnel from your localhost to host1, and then from host1 to host2. Provided that host1 and host2 both have SSH servers, you can join two tunnels in the following way:

ssh -L 9999:host2:22 host1user@host1
ssh -L 9998:localhost:5900 -p 9999 host2user@localhost

The first tunnel connects the SSH server on host2 port 22 to localhost port 9999 (using host1's SSH server). The second tunnel connects the service at port 5900 on host2 to the localhost port 9998 (through host2's SSH server available now as a result of the first tunnel).

So when whatever client application is pointed to the localhost port 9998, it's really tunneling to host2 port 5900 through host2's SSH server... where the connection to host2's SSH server is actually through another tunnel (from localhost port 9999 to host2 port 22 over host1's SSH server). And there you have it: a tunnel within a tunnel. My head hurts.

Stream iTunes (via1, via2)

This gets a little trickier because of the way iTunes advertises itself on the network. We'll need to use another application to assist with that bit, but the tunneling is the same. iTunes streams over port 3869, so:

ssh -L 9999:itunesserver:3689 sshuser@sshserver

Then, with the tunnel up and running,

dns-sd -P "Home iTunes" _daap._tcp local 3689 localhost.local. 127.0.0.1 "Arbitrary text record"

Package that up into a script, and you're good to go.

Cygwin Notes

Just some notes.

• Missing the "clear" terminal command? Install the "ncurses" package.
• Always grab "openssh"