February 2008 - Posts

foreach (IPAddress ip in Dns.GetHostEntry(Dns.GetHostName()).AddressList)
     Console.WriteLine(ip.ToString());

So in my previous post, I set the following goals for my MyBook WE2:

  1. Serve as a storage point for all files I wanted to manually backup.
  2. Automatically download torrents from a directory I would put torrents into
  3. Automatically download predefined lists of videos and convert them to various formats
  4. Consume schedule video from my PCTV2GO and convert it to various formats
  5. Download predefined lists of blogs etc I wanted to read and convert them to audio.
  6. Serve as a print server? Maybe print out predefined documents for consumption.

Let's see what we can get accomplished. We'll start with the first goal and get our storage point setup. I want to have this mirrored so I know my files are safe from a hard drive crash. I'm not terribly worried about using my my 500GB of space yet since I haven't had anything near that much space on my laptops and I have always just deleted items as I went along. There is more than enough content on the internet that I look at daily/weekly to replace what I didn't see, so I'll just cycle it out. So first, I simply logged into the web interface by going to the ip address of the device. I then went to the 'Drive Management' tab, click on "Change Drive Type", and change my drive type to "Secure volume (RAID-1)". I get a bunch of warnings which I go through and then I wait. This process takes about an hour or so. I just went out and when I came back it was ready.

After reading a lot of forums and blog posts about this device, i decided I didn't really have any interest in the 'mionet' software or the ability to view all of my files from anywhere. At least not with their software. Perhaps serving them via http isn't out of the question, but it's not on the list and not a priority. I mention this because mionet is one of the ways you access your files, and you are supposed to pop in the Western Digital CD that comes with the drive and install all their stupid software. I'm sure it's useful to someone, but Samba is already installed and running by default, so if you have client for ms attached to your nic, which most windows users do, you can just go to \\<ip of your mybook\PUBLIC to view the public share that is on the MyBook by default. This worked great for me. I will merely use sub-folders and permissions from the command line on the box to manage security. So I create a few folders after attaching to the UNC and move a bunch of music/pictures/movies/documents/etc over. This is probably where I ran into my first sad moment. The data transmission wasn't exactly 'blazing fast' as I had hoped. Perhaps 'blazing' isn't what i really expected, but it took a lot longer than I thought. I believe moving about 8GB took probably an hour or 90 minutes. This was another one of those things where I just left for a while and came back. The estimate was a lot and I wasn't gone more than 90 minutes though. Now, this can probably mostly be attributed to the fact that I'm on a simple wireless G network. The MyBook comes equipped with a gigabit nic, however my little wireless hub is only 100 and over standard G wireless, you are limited a lot more.

So, at this time, we need to do a little basic security. Back in the web interface again, I go to the 'General Setup' tab and change the admin password. This is where I ran into what I think was my MOST dissapointing problem with this device. Entering a strong password I regularly use returns this happy message: "Use only letters, numbers, a-z or 0-9".  Oh, how sad. no symbols? Yuck! Ah, well, who knows if I'll even end up using this interface in the end.

Well,  at this point, we *have* accomplished our very first goal. The device does "serve as a storage point for all files I wanted to manually backup."

I've decided to add two more goals to my list:

  1. Automate personal administrative tasks.
  2. sync my backup data with my s3 account.
Number one is in reference to servicing the many generic daily tasks i can automate with scripting. I could purchase any box to do these for me, but I'd like to have them run off of this one little device. This will probably be accomplished via some of the other goals, but I think it is a distinct bullet point itself. Two is something I've wanted to do for a long while. I have a large majority of what I consider 'important personal data' backed up to s3 already. Really pictures/movies of my kids, family, etc. I do have them backed up on some DVDs around here as well, but I would like it to be organized digitally, then simply stored on Amazon's S3 for offsite. Storage. Perhaps, eventually, incorporating this with a formalized plan using DVDs or whatever we're burning to at that time. Hard drives are cheap enough, it seems a waste to burn to media, but who knows, and that's in the future and has nothing to do with accomplishing the goal really. =P

More to come, that's enough for now. The fun has barely started!!! =D 

 

 

Had a burning need for some linux loving recently. I'll post about that shortly too. I found a great linux command line cheat sheet at http://www.pixelbeat.org/cmdline.html

CommandDescription
apropos whatisShow commands pertinent to string. See also threadsafe
man -t man | ps2pdf - > man.pdfmake a pdf of a manual page
 which commandShow full path name of command
 time commandSee how long a command takes
time catStart stopwatch. Ctrl-d to stop. See also sw
nice infoRun a low priority command (The "info" reader in this case)
renice 19 -p $$Make shell (script) low priority. Use for non interactive tasks
dir navigation
cd -Go to previous directory
cdGo to $HOME directory
 (cd dir && command)Go to dir, execute command and return to current dir
pushd .Put current dir on stack so you can popd back to it
alias l='ls -l --color=auto'quick dir listing
ls -lrtList files by date. See also newest and find_mm_yyyy
ls /usr/bin | pr -T9 -W$COLUMNSPrint in 9 columns to width of terminal
 find -name '*.[ch]' | xargs grep -E 'expr'Search 'expr' in this dir and below. See also findrepo
 find -type f -print0 | xargs -r0 grep -F 'example'Search all regular files for 'example' in this dir and below
 find -maxdepth 1 -type f | xargs grep -F 'example'Search all regular files for 'example' in this dir
 find -maxdepth 1 -type d | while read dir; do echo $dir; echo cmd2; doneProcess each item with multiple commands (in while loop)
find -type f ! -perm -444Find files not readable by all (useful for web site)
find -type d ! -perm -111Find dirs not accessible by all (useful for web site)
locate -r 'file[^/]*\.txt'Search cached index for names. This re is like glob *file*.txt
look referenceQuickly search (sorted) dictionary for prefix
grep --color reference /usr/share/dict/wordsHighlight occurances of regular expression in dictionary
archives and compression
 gpg -c fileEncrypt file
 gpg file.gpgDecrypt file
 tar -c dir/ | bzip2 > dir.tar.bz2Make compressed archive of dir/
 bzip2 -dc dir.tar.bz2 | tar -xExtract archive (use gzip instead of bzip2 for tar.gz files)
 tar -c dir/ | gzip | gpg -c | ssh user@remote 'dd of=dir.tar.gz.gpg'Make encrypted archive of dir/ on remote machine
 find dir/ -name '*.txt' | tar -c --files-from=- | bzip2 > dir_txt.tar.bz2Make archive of subset of dir/ and below
 find dir/ -name '*.txt' | xargs cp -a --target-directory=dir_txt/ --parentsMake copy of subset of dir/ and below
 ( tar -c /dir/to/copy ) | ( cd /where/to/ && tar -x -p )Copy (with permissions) copy/ dir to /where/to/ dir
 ( cd /dir/to/copy && tar -c . ) | ( cd /where/to/ && tar -x -p )Copy (with permissions) contents of copy/ dir to /where/to/
 ( tar -c /dir/to/copy ) | ssh -C user@remote 'cd /where/to/ && tar -x -p' Copy (with permissions) copy/ dir to remote:/where/to/ dir
 dd bs=1M if=/dev/sda | gzip | ssh user@remote 'dd of=sda.gz'Backup harddisk to remote machine
rsync (Use the --dry-run option for testing)
 rsync -P rsync://rsync.server.com/path/to/file fileOnly get diffs. Do multiple times for troublesome downloads
 rsync --bwlimit=1000 fromfile tofileLocally copy with rate limit. It's like nice for I/O
 rsync -az -e ssh --delete ~/public_html/ remote.com:'~/public_html'Mirror web site (using compression and encryption)
 rsync -auz -e ssh remote:/dir/ . && rsync -auz -e ssh . remote:/dir/Synchronize current directory with remote one
ssh (Secure SHell)
 ssh $USER@$HOST commandRun command on $HOST as $USER (default command=shell)
ssh -f -Y $USER@$HOSTNAME xeyesRun GUI command on $HOSTNAME as $USER
 scp -p -r $USER@$HOST: file dir/Copy with permissions to $USER's home directory on $HOST
 ssh -g -L 8080:localhost:80 root@$HOSTForward connections to $HOSTNAME:8080 out to $HOST:80
 ssh -R 1434:imap:143 root@$HOSTForward connections from $HOST:1434 in to imap:143
wget (multi purpose download tool)
(cd cmdline && wget -nd -pHEKk http://www.pixelbeat.org/cmdline.html)Store local browsable version of a page to the current dir
 wget -c http://www.example.com/large.fileContinue downloading a partially downloaded file
 wget -r -nd -np -l1 -A '*.jpg' http://www.example.com/dir/Download a set of files to the current directory
 wget ftp://remote/file[1-9].iso/FTP supports globbing directly
wget -q -O- http://www.pixelbeat.org/timeline.html | grep 'a href' | headProcess output directly
 echo 'wget url' | at 01:00Download url at 1AM to current dir
 wget --limit-rate=20k urlDo a low priority download (limit to 20KB/s in this case)
 wget -nv --spider --force-html -i bookmarks.htmlCheck links in a file
 wget --mirror http://www.example.com/Efficiently update a local copy of a site (handy from cron)
networking (Note ifconfig, route, mii-tool, nslookup commands are obsolete)
 ethtool eth0Show status of ethernet interface eth0
 ethtool --change eth0 autoneg off speed 100 duplex fullManually set ethernet interface speed
 iwconfig eth1Show status of wireless interface eth1
 iwconfig eth1 rate 1Mb/s fixedManually set wireless interface speed
iwlist scanList wireless networks in range
ip link showList network interfaces
 ip link set dev eth0 name wanRename interface eth0 to wan
 ip link set dev eth0 upBring interface eth0 up (or down)
ip addr showList addresses for interfaces
 ip addr add 1.2.3.4/24 brd + dev eth0Add (or del) ip and mask (255.255.255.0)
ip route showList routing table
 ip route add default via 1.2.3.254Set default gateway to 1.2.3.254
tc qdisc add dev lo root handle 1:0 netem delay 20msecAdd 20ms latency to loopback device (for testing)
tc qdisc del dev lo rootRemove latency added above
host pixelbeat.orgLookup DNS ip address for name or vice versa
hostname -iLookup local ip address (equivalent to host `hostname`)
whois pixelbeat.orgLookup whois info for hostname or ip address
netstat -tuplList internet services on a system
netstat -tupList active connections to/from system
windows networking (Note samba is the package that provides all this windows specific networking support)
smbtreeFind windows machines. See also findsmb
 nmblookup -A 1.2.3.4Find the windows (netbios) name associated with ip address
 smbclient -L windows_boxList shares on windows machine or samba server
 mount -t smbfs -o fmask=666,guest //windows_box/share /mnt/shareMount a windows share
 echo 'message' | smbclient -M windows_boxSend popup to windows machine (off by default in XP sp2)
text manipulation (Note sed uses stdin and stdout, so if you want to edit files, append <oldfile >newfile)
 sed 's/string1/string2/g'Replace string1 with string2
 sed 's/\(.*\)1/\12/g'Modify anystring1 to anystring2
 sed '/ *#/d; /^ *$/d'Remove comments and blank lines
 sed ':a; /\\$/N; s/\\\n//; ta'Concatenate lines with trailing \
 sed 's/[ \t]*$//'Remove trailing spaces from lines
 sed 's/\([\\`\\"$\\\\]\)/\\\1/g'Escape shell metacharacters active within double quotes
seq 10 | sed "s/^/      /; s/ *\(.\{7,\}\)/\1/"Right align numbers
 sed -n '1000p;1000q'Print 1000th line
 sed -n '10,20p;20q'Print lines 10 to 20
 sed -n 's/.*<title>\(.*\)<\/title>.*/\1/ip;T;q'Extract title from HTML web page
 sort -t. -k1,1n -k2,2n -k3,3n -k4,4nSort IPV4 ip addresses
echo 'Test' | tr '[:lower:]' '[:upper:]'Case conversion
tr -dc '[:print:]' < /dev/urandomFilter non printable characters
history | wc -lCount lines
set operations (Note you can export LANG=C for speed. Also these assume no duplicate lines within a file)
 sort file1 file2 | uniqUnion of unsorted files
 sort file1 file2 | uniq -dIntersection of unsorted files
 sort file1 file1 file2 | uniq -uDifference of unsorted files
 sort file1 file2 | uniq -uSymmetric Difference of unsorted files
 join -a1 -a2 file1 file2Union of sorted files
 join file1 file2Intersection of sorted files
 join -v2 file1 file2Difference of sorted files
 join -v1 -v2 file1 file2Symmetric Difference of sorted files
math
echo '(1 + sqrt(5))/2' | bc -lQuick math (Calculate φ). See also bc
echo 'pad=20; min=64; (100*10^6)/((pad+min)*8)' | bcMore complex (int) e.g. This shows max FastE packet rate
echo 'pad=20; min=64; print (100E6)/((pad+min)*8)' | pythonPython handles scientific notation
echo 'pad=20; plot [64:1518] (100*10**6)/((pad+x)*8)' | gnuplot -persistPlot FastE packet rate vs packet size
echo 'obase=16; ibase=10; 64206' | bcBase conversion (decimal to hexadecimal)
echo $((0x2dec))Base conversion (hex to dec) ((shell arithmetic expansion))
units -t '100m/9.74s' 'miles/hour'Unit conversion (metric to imperial)
units -t '500GB' 'GiB'Unit conversion (SI to IEC prefixes)
units -t '1 googol'Definition lookup
seq 100 | (tr '\n' +; echo 0) | bcAdd a column of numbers. See also add and funcpy
calendar
cal -3Display a calendar
cal 9 1752Display a calendar for a particular month year
date -d friWhat date is it this friday. See also day
date --date='25 Dec' +%AWhat day does xmas fall on, this year
date --date '1970-01-01 UTC 2147483647 seconds'Convert number of seconds since the epoch to a date
TZ=':America/Los_Angeles' dateWhat time is it on West coast of US (use tzselect to find TZ)
 echo "mail -s 'get the train' P@draigBrady.com < /dev/null" | at 17:45Email reminder
echo "DISPLAY=$DISPLAY xmessage cooker" | at "NOW + 30 minutes"Popup reminder
locales
printf "%'d\n" 1234Print number with thousands grouping appropriate to locale
BLOCK_SIZE=\'1 ls -lget ls to do thousands grouping appropriate to locale
echo "I live in `locale territory`"Extract info from locale database
LANG=en_IE.utf8 locale int_prefixLookup locale info for specific country. See also ccodes
locale | cut -d= -f1 | xargs locale -kc | lessList fields available in locale database
recode (Obsoletes iconv, dos2unix, unix2dos)
recode -l | lessShow available conversions (aliases on each line)
 recode windows-1252.. file_to_change.txtWindows "ansi" to local charset (auto does CRLF conversion)
 recode utf-8/CRLF.. file_to_change.txtWindows utf8 to local charset
 recode iso-8859-15..utf8 file_to_change.txtLatin9 (western europe) to utf8
 recode ../b64 < file.txt > file.b64Base64 encode
 recode /qp.. < file.txt > file.qpQuoted printable decode
 recode ..HTML < file.txt > file.htmlText to HTML
recode -lf windows-1252 | grep euroLookup table of characters
echo -n 0x80 | recode latin-9/x1..dumpShow what a code represents in latin-9 charmap
echo -n 0x20AC | recode ucs-2/x2..latin-9/xShow latin-9 encoding
echo -n 0x20AC | recode ucs-2/x2..utf-8/xShow utf-8 encoding
CDs
 gzip < /dev/cdrom > cdrom.iso.gzSave copy of data cdrom
 mkisofs -V LABEL -r dir | gzip > cdrom.iso.gzCreate cdrom image from contents of dir
 mount -o loop cdrom.iso /mnt/dirMount the cdrom image at /mnt/dir (read only)
 cdrecord -v dev=/dev/cdrom blank=fastClear a CDRW
 gzip -dc cdrom.iso.gz | cdrecord -v dev=/dev/cdrom -Burn cdrom image (use dev=ATAPI -scanbus to confirm dev)
 cdparanoia -BRip audio tracks from CD to wav files in current dir
 cdrecord -v dev=/dev/cdrom -audio *.wavMake audio CD from all wavs in current dir (see also cdrdao)
 oggenc --tracknum='track' track.cdda.wav -o 'track.ogg'Make ogg file from wav file
disk space (See also FSlint)
ls -lSrShow files by size, biggest last
du -s * | sort -k1,1rn | headShow top disk users in current dir. See also dutop
df -hShow free space on mounted filesystems
df -iShow free inodes on mounted filesystems
fdisk -lShow disks partitions sizes and types (run as root)
rpm -q -a --qf '%10{SIZE}\t%{NAME}\n' | sort -k1,1nList all packages by installed size (Bytes) on rpm distros
dpkg-query -W -f='${Installed-Size;10}\t${Package}\n' | sort -k1,1nList all packages by installed size (KBytes) on deb distros
dd bs=1 seek=2TB if=/dev/null of=ext3.testCreate a large test file (taking no space). See also truncate
monitoring/debugging
tail -f /var/log/messagesMonitor messages in a log file
strace -c ls >/dev/nullSummarise/profile system calls made by command
strace -f -e open ls >/dev/nullList system calls made by command
ltrace -f -e getenv ls >/dev/nullList library calls made by command
lsof -p $$List paths that process id has open
lsof ~List processes that have specified path open
tcpdump not port 22Show network traffic except ssh. See also tcpdump_not_me
ps -e -o pid,args --forestList processes in a hierarchy
ps -e -o pcpu,cpu,nice,state,cputime,args --sort pcpu | sed '/^ 0.0 /d'List processes by % cpu usage
ps -e -orss=,args= | sort -b -k1,1n | pr -TW$COLUMNSList processes by mem usage. See also ps_mem.py
ps -C firefox-bin -L -o pid,tid,pcpu,stateList all threads for a particular process
ps -p 1,2List info for particular process IDs
last rebootShow system reboot history
free -mShow amount of (remaining) RAM (-m displays in MB)
watch -n1 'cat /proc/interrupts'Watch changeable data continuously
system information (see also sysinfo) ('#' means root access is required)
uname -aShow kernel version and system architecture
head -n1 /etc/issueShow name and version of distribution
cat /proc/partitionsShow all partitions registered on the system
grep MemTotal /proc/meminfoShow RAM total seen by the system
grep "model name" /proc/cpuinfoShow CPU(s) info
lspci -tvShow PCI info
lsusb -tvShow USB info
mount | column -tList mounted filesystems on the system (and align output)
#dmidecode -q | lessDisplay SMBIOS/DMI information
#smartctl -A /dev/sda | grep Power_On_HoursHow long has this disk (system) been powered on in total
#hdparm -i /dev/sdaShow info about disk sda
#hdparm -tT /dev/sdaDo a read speed test on disk sda
#badblocks -s /dev/sdaTest for unreadable blocks on disk sda
interactive (see also linux keyboard shortcuts)
readlineLine editor used by bash, python, bc, gnuplot, ...
screenVirtual terminals with detach capability, ...
mcPowerful file manager that can browse rpm, tar, ftp, ssh, ...
gnuplotInteractive/scriptable graphing
linksWeb browser
miscellaneous
alias hd='od -Ax -tx1z -v'Handy hexdump. (usage e.g.: • hd /proc/self/cmdline | less)
alias realpath='readlink -f'Canonicalize path. (usage e.g.: • realpath ~/../$USER)
set | grep $USERSearch current environment
 touch -c -t 0304050607 fileSet file timestamp (YYMMDDhhmm)

 

 

 

So, with the recent announcement of CompUSA's closing, I have  been swinging by there on occasion to check out the deals on whatever stuff there is. I recently purchased a MyBook World Edition II 1TB model and have been having a lot of fun with it. They were down to 299 which isn't ridiculously cheap, but it's far, far below the retail of... whatever it was. Something crazy like 450 i think. I found them several places online for around 320ish. I went by the other day and they are now down to 257 which *is* a pretty sweet deal.

So, I thought I would read up about my purchase and it turns out that this device is actually quite configurable. It is equipped with ~200MHz processor and 32MB of ram.\ from what I have read online. 'cat /proc/cpuinfo' shows the processor as a ARM926EJ-Sid(wb) rev 5 (v5l) and meminfo shows 30032 Total. So, definitely not winning any speed awards, but more than adequate for what it's shipped to accomplish.

With my major goal accomplished, I had some hope that I would be able to 'hack' the device a little. A very little bit of searching on google revealed this page, among many others. 'cat /proc/version' shows I'm running Linux version 2.6.17.14 (clarke@mccoy.oxsemi.com.asic.oxsemi.com) (gcc version 4.1.0) #1 PREEMPT Mon Mar 26 12:36:33 BST 2007. Armed with my little research, I decided I had the following goals for this device:

  1. Serve as a mirrored storage point for all files I wanted to manually backup.
  2. Automatically download torrents from a directory I would put torrents into
  3. Automatically download predefined lists of videos and convert them to various formats
  4. Consume schedule video from my PCTV2GO and convert it to various formats
  5. Download predefined lists of blogs etc I wanted to read and convert them to audio.
  6. Serve as a print server? Maybe print out predefined documents for consumption.
As you can see, a lot of this revolves around creating custom media to consume. I'm not a big fan of pirated video/music. I personally buy all my DVD and CD media I want to consume. I do believe that torrenting broadcast television is not illegal, however I want to stream the video from my PCTV2GO for my broadcast shows and record them from there. Once this is complete, I have thought about purchasing something with a little more streaming power to perform something similar to what Orb does. I'm not sure I want to make that kind of investment, especially if I can have this converting all of the information I want to consume in ways that are friendly to me and my devices. I'll let you know how it goes. =D

PS. I recently spoke to a manager at my local CompUSA about this announcement and confirmed both of the Jacksonville, FL stores will remain open under the new deal. 

If you've never used the 'for' command in windows, it is your friend. It longs for you to use it! It is 'Da Bizzle' as the kids say. Here's a silly little example of using it to delete all of the temporary internet files from all of the profiles on a machine.

cd C:\Documents and Settings
for /f "tokens=*" %i in ('dir /b') do rd /s ".\%i\Local Settings\Temporary Internet Files\"

Curved monitor happiness

Thanks Alienware/Dell! =)

 

If you haven't heard about it, Amazon had an outtage the other day with some of it's shared services. Here's a great article explaining "what up with that"

http://www.roughtype.com/archives/2008/02/why_s3_failed.php 

Here's a quote from Amazon about what happened:

Early this morning, at 3:30am PST, we started seeing elevated levels of authenticated requests from multiple users in one of our locations. While we carefully monitor our overall request volumes and these remained within normal ranges, we had not been monitoring the proportion of authenticated requests. Importantly, these cryptographic requests consume more resources per call than other request types.

Shortly before 4:00am PST, we began to see several other users significantly increase their volume of authenticated calls. The last of these pushed the authentication service over its maximum capacity before we could complete putting new capacity in place. In addition to processing authenticated requests, the authentication service also performs account validation on every request Amazon S3 handles. This caused Amazon S3 to be unable to process any requests in that location, beginning at 4:31am PST. By 6:48am PST, we had moved enough capacity online to resolve the issue.

 I've been using many of amazon's projects for off and on developmental projects pretty much since they came out and have always found their services reliable and pretty easy to use. 

Those of you that know me personally are aware I'm an amateur guitar player. I started playing a little over a year ago and it's been a great experience. A couple months ago, Gibson released the Robot Guitar

Gibson Robot Guitar

 

Here's a little blurb from the Gibson site:


Since the dawn of the instrument, musicians have come to accept the guitar's imperfections and lack of tonal precision as necessary evils. Onstage and off, guitarists have fought to stay in tune. Every music lover and performer has had to suffer through the show—halting, mood-killing atonal droning of a loudly amped guitar being brought into tune. And in the studio, or at home, imprecise intonation throws the guitar in and out of tune, up and down the neck, as the instrument requires tweaking with each season and at times with each string change.

For as long as the guitar has existed, guitar lovers have had two choices—live with a temperamental and out-of-tune instrument or make frequent trips to the shop for setups. Not anymore.

Introducing the Gibson Robot Guitar. All you have to do is play it. 

 

 

Gibson Robot Guitars

 

This guitar has been awesome. Sadly, the pull out button that you use to do all the fun stuff came off the other day. I took it to the shop and the guitar repair guy had sadly (for me) gone to a Van Halen concert, which was good for him. =P Further bad luck for me is that it is a holiday weekend and the repair shop is closed until Tuesday. They assured me they would take care of it for me on Tuesday first thing so I will be back up there then and I'll let everyone know how my first interaction with Gibson's customer service and Guitar Center's repair shop goes. =) I expect great things based on my trips to their shop.

 Some reviews if you are interested.