Photograph Digitising and Preservation

Perhaps as you get older, and certainly when you have children, you start thinking a bit more about preserving family photos. All photos of my kids have always been digital, so for me, preserving and archiving that has been a digital storage and backup dilemma, so have a look down the bottom of this post for my solutions on that.

However, almost all of my own childhood pictures are on single pieces of paper in boxes, some in albums, and some of which we still have the single negatives of. For me, that’s an issue as I live on the other side of the planet from the rest of my family and those photos, and there’s only that single copy, which we’d like to duplicate to have here in Japan as we start our own family, and not just to reduce that single point of disaster. I want to give my kids a giggle when they look at what I did when I was a kid.

Most of my old family photos were shot on cheap 110 film cameras with those stick flashes which melted after one use (my children will laugh at this post when they’re old enough to read it!)

110 film was widely available in the seventies and eighties, and roundly regarded as not very good nowadays. This means many of my childhood photos aren’t great technically, but from going through this process with a hundred or so photos so far, it isn’t the image quality that matters, it’s the memory of the moment, so don’t worry if that awesome memory you have in your head seems to have a coffee ring on the corner of the print and is slightly out of focus on the paper – it doesn’t detract at all.

Scanning

I have a recent, but not new, Epson GT-X770 scanner – it’s a mid range home scanner, and supports slide and negative scanning via a top lid light, and plastic negative holders. I did though, have to make my own holder for 110 film negative, as virtually no scanners support it natively. It also has some solid scan drivers and hardware elements for colour restoration, scratch removal etc.. I tend to scan two versions of each print; one a straight un-modded one, and one with colour restored because I actually find the scanner seems to do a better job of this that Photoshop for some reason.

The resolution I scan at depends on the source and the photo. There are plenty of guides out there, though I find some to be a bit off to me – a good website for reviewing some of the more technical aspects is ScanTips

Generally I scan colour paper print photos at 300dpi, and a few at 600dpi [dots per inch].

300dpi is easily good enough for most prints, and recently, what most photo print shops printed at anyway. From what I can tell from testing on the 110 prints, 300dpi is already far above what was captured on the paper anyway. I use 600dpi for some black and white prints from good sources, though again, I’m probably going too high and am pretty much capturing high resolution grain some of the time.

For negatives, you’ll use a much higher dpi setting (as the negatives are so small) and you start to see why scanners often have fantastically high scan numbers.

I tend to go for ~2400-3200dpi, but again, you run the risk of purely scanning grain in the negative, rather than capturing any real value, so no need to put the scanner on max, unless you have pristine, well shot slides or negatives. Again, for me, my sources are relatively poor, so I don’t need to go too far – to around 3200dpi for some 35mm negatives I had from early backpacking trips with my old Olympus.

Other Scanner Settings
I tend to just try to capture everything in the scanner, and then ‘fix’ in software on a copy. The only exception is really colour restore as I said, and some hardware features. Unsharp mask etc. I just leave for software adjustments later on.

File Format

When saving anything you’ve digitized from an analogue source, you want it to be in a loss-less format – this means that it’s exactly what you captured. The other system is ‘lossy’ and generally you don’t see the difference, except when you do multilpe generations of saves.

I save the files as loss-less TIFF files. This seems to be the most widely supported format, and holds comments and other things fairly well, it also has some lossless compression options, and handles high colour depth for those with higher end scanners. I’ve saved some of my own scans as PNG format also, as there’s nothing wrong with it, and has some advantages over TIFF in loss-less compression, but lacks support in some applications.

Never, ever, save as JPG – just don’t – always try to save a loss-less version as your ‘gold master’, and take JPGs from that. For any file conversion, I recommend IrfanView for Windows, and GraphicConverter for OS X.

Outsourcing

If the thought of getting a scanner and taking the time and effort to do all those photos seems scary or just plain painful, you can pay someone to scan your prints and negatives, and return them to you. This might also be an option if you’re looking at a huge archive.

There was some controversy a few years ago that many of the companies ship your photos to India for scanning and clean up, though there never seemed to be anyone who’d lost photos or had any other issues with any of these services, and as long as the end result is good, I fail to see how it’s an issue.

That Digital Storage
Backups for my computers used to really be about things I’d written and so on, but basically, like this blog – I always have a copy online these days, but I will confess I still backup a WordPress export locally now and then, but I think that’s relatively safe.

Prints are on paper, and you have a negative. There’s no worry about obsolete file formats, or applications, you just look at them in an album – risk of deletion is fairly low.

For any format though, there’s the risk of natural disaster, fires, theft and all manner of things – I tragically saw a lot of this when I went up to Tohoku to help clean up the tsunami damage – we all kept a keen eye out for any photos, CDs, hard drives or negatives which may help someone put their family memories back together.

I see a lot of people saying to keep a USB drive as a backup, but I’m not sure that’s safe enough. I believe in 3 copies – two different media locally, and one off-site copy. For me, most of my photos are on a Mac, that’s backed up sort-of live to a USB drive (not a portable one) on my desk via Time Machine, and then I use CrashPlan to upload another copy to their servers over the internet.

I used to back up to DVDs, but the libraries became just too large to burn, and I can’t trust small hard drives, though for a while i would keep one in my office drawer as a backup.

So that’s my setup for archiving the analogue photographic past. There are cheaper, simpler, or more expensive ways of doing it, but this is working for me. I don’t think you can go too wrong as long as you get a basic loss-less image file at a decent resolution, and back it up – and keep that original print or negative.

Curse you, MX records! Sorry, my fault.

It’s been a while since I put a techie post up on here – shame on me! Here’s one about me doing something stupid.

Like many, I have a couple of email addresses. Actually, I have a silly amount I’ve set up to play with services over the years, but I’ve been slowly culling them. Anyway, I basically have two main addresses now – one which I use day to day for sign-ups, and one based on a domain name I own, and is mainly for friends and family which is now quite a few years old.

That, for a while now has been run from Google Apps, a relatively under appreciated offering from Google as far as I can tell. Essentially what it is, is a Gmail and Google Apps backend for your own company/domain, so all my mail for that address comes and goes from my address @brightblack.net account, but is purely going through Gmail and gets all the benefits of the spam filter and 7GB+ of storage, for free. I like it – so much better than a hotmail account or an ISP namespace.

My problem actually started a few weeks ago, but I didn’t notice as it’s not a high volume account, but I wasn’t getting any new mail. Then, last week my parents mentioned they’d actually got a bounced reply. I actually replicated it straight off from my phone thanks to Softbank’s insane timeout setting:

Action: failed Status: 5.2.0 Remote-MTA: dns; brightblack.net (___.___.___.___) Diagnostic-Code: smtp; 554 5.7.1 : Relay access denied

(IP address obscured there!)

Odd. So I logged in to my Google Apps account, nope, all looked fine there, but no new mail. So I went further up the chain. My domain registrar is called PairNic, who have always been good to deal with, and the www.brightblack.net domain pointer seemed fine. So the next logical question is: what had I changed with this domain – any aspect of it – lately?

Answer: I’d re-set it to point to this blog … a few weeks ago.

Ah.

That’s a coincidence.

From Google Apps, I did a quick MX records test, where it basically does a DNS lookup (of which MX records are a component) on my domain and as somewhat expected, no MX records found. Well that explains why I wasn’t getting mail and people were getting bounced. As far as the internets were concearned, I wasn’t there.

So, I logged in to Pairnic again, went into their DNS/email config, and lo and behold – nothing – so a quick check at both WordPress and Google apps showed me my MX records should look something like:

Priority          Mail Server
1           ASPMX.L.GOOGLE.COM.
5           ALT1.ASPMX.L.GOOGLE.COM.
5           ALT2.ASPMX.L.GOOGLE.COM.
10          ASPMX2.GOOGLEMAIL.COM.
10         ASPMX3.GOOGLEMAIL.COM.

So I entered that into Pairnic’s pages, waited only a few minutes for it to replicate, and then the test from Google showed I had MX records again, and a quick email from my phone successfully arrived, followed over the next day by a small deluge as spooled messages came in.

You can actually get some good, simple steps to troubleshoot this in the Google Apps Tech Support area, as I discovered mostly after I’d fixed it.

So how did it break? As I suspected, it was when I re-pointed my old  brightblack website address to this blog through PairNic, I missed a large amount of red warning text saying exactly what would happen:

Warning:  Enabling any Forwarding features (E-mail or Web site Forwarding or Parking) will remove any previous DNS settings that you may have had with your account. Any previous nameserver, DNS, or e-mail settings will be lost.

How I missed that, I’ll never know – PairNic obviously make it pretty clear, but I must have thought that since I was only playing with NS (website) settings really, my email would be unchanged. I should have re-read that! All in all, I got off lucky – it took me less than an hour in front of the computer to realise what an oaf I’d been and to fix and test it – looking at some of the Google forums, some people have much larger issues.

If nothing else, I’ve refreshed my rusty knowledge of DNS, MX records and how my own email setup actually works, which is no bad thing. It’s also got me delving a little more into parts of DNS I hadn’t really looked at, like priorities, CNAMES, SPF records (sender-permitted-from), and gain a bit more knowledge on the subject of email in general, and some of the changes coming with IPv6!

Read the red text. Twice.

[Not] Unlimited Mozy

Something interesting happened on February 1st. in the incredibly exciting realm of online backups. Let’s just address one thing first: backups. Please make backups and regularly. Even better, have an onsite and an offsite backup. For me, offsite used to mean burning to DVD-R and shipping to the parents, nine time zones away, and thus preserving all my family’s pictures, video and a few other bits in case something Bad (™) happened.

Initially I went with JungleDisk/Amazon S3 for this, but in November 2009 shifted to Mozy – unlimited backup for 5USD a month which made more sense for me and meant I could also backup all the .dv videos and music, then about 140GB in total, now about 175GB.

As I say then, on Feb. 1st. Mozy dropped their unlimited plan, and implemented a tier based system. Instead of unlimited for 5 USD/month, it would be 50GB for 6USD/month and additional fees per 20GB above that. For me then, that would be about 21USD / month – a decent price increase.

As I posted on their forums, on the now 847 post complaint thread:

“I appreciate storage (even with EMC behind you) costs money. Bandwidth costs money. I can understand your move to a tiered, non unlimited plan, I really can.  That said, here’s my two thoughts:

- you set the baseline too low – I see a lot of people on Twitter, and myself between 150-200GB, and for us, there’s just no economic reason to stay once our plans expire. For me that’s 10months, though I need to be careful to avoid the auto-renewal if I decide to leave (and unfortunately take my parents with me). I don’t think we’re abusing your generous unlimited offer with that amount per month. My fee likely covers my cost to you.

- but honestly, what irks me most is that I learn about this on Twitter, and on Lifehacker. I don’t have an e-mail from you, I don’t see anything on the blog from you. That just seems to imply you’re not interested in customer relations. As I implied above, that I only get mail from Mozy after they’ve charged my credit card, but not for a product offering shift like this means that I think you’re going to have a painful reaction from users who don’t watch every product announcement not in their mailbox.

Again, it’s a bold move by Mozy and I’m sure others will follow, and users will chose to stay or go, but really, whoever handled the rollout of this announcement needs to be given a little talking to.”

Continue reading

App notes: Instagr.am

Just a quick post here for an iPhone app I’ve been playing with this week. Up front: this isn’t going to change your life. However, it is a bit of fun – instagr.am. It’s a free app which basically takes photos with the iPhone camera, but allows you to apply to some filters and such to make them look quite interesting, and then to add an extra feature in comparison to the other apps which do that, it allows you to very simply send them to several social networks from within the apps such as Facebook, Flickr and Twitter. If you set the GPS, you can check in to Foursquare there. Unfortunately, it doesn’t support WordPress yet.

It is kind of fun. Here’s a couple I’ve done over the last week, and if you do post from your phone regularly, it might add a bit more atmosphere. Of the currently available filters, I quite like the ’1977′ one, since of late I’ve been scanning some old family pictures from that time, and they do indeed now look pretty like that, so I took a picture of my battered iPod Classic in that format; I quite like it.

 

old ipod

An iPod from 1977.

A Coffee at Denny's

A Coffee at Denny's

My TechNet at Home

Recently I just paid ~21,000yen for a one year MS TechNet Standard subscription for myself, for usage at home.

I’m not at all sure if this is common or whether people still think Technet is just for business – or even if many people have even heard of it. Essentially though, this gets me access to most MS OSs and their variants (32-/64- bit, languages etc.) e.g. XP, Vista, Windows 7 and several server versions. It also allows me to use various versions of Office and other applications. There are restrictions: they have to be my PCs, and for non commercial usage and so on. That works for me.

I actually had several reasons for paying for this, so I just wanted to go over them here.
Firstly, I ama tech person; I use computers a lot. Whilst I use a Mac and Linux, I also use Windows a lot for various things, and I build my own desktops, so I don’t buy PCs with Windows pre-installed. I therefore want to use the latest OSs and play with different applications. I’m also a user of the less famous Office apps like Visio and Project, and they’re included with TechNet Standard which is nice. It also means I can try out various servers like SQL servers in virtual machines for edification.

Another of my niche requirements, is because I live in Japan, which affects me in two ways; firstly, I want a PC which can run in Japanese and English mode at OS and application level. Mac and Linux do this out of the box, but for some reason, Microsoft elects to achieve this via multi user interfaces, MUIs, which are separate downloads from the base OS.

In Windows 7, these are at least available from the Windows Update window, but only on select versions, and for consumers, that’s essentially Windows 7 Ultimate, which carries a retail price of ~230 USD. That’s a lot. In fact, it pays for the technet subscription on its own, and that’s before we allow ~400 USD for Office, or even 100 USD for the Home version if I forget Visio and Project.

There’s then the issue of if I buy a laptop here in Japan – most don’t come with English OS options, though some can be configured with an English keyboard – sorry Japan keyboard fans, I love a big spacebar. So then, I’d be looking at at least a copy of Home Premium at ~ 140 USD.

So for straight economics, it’s a winning solution really, even if only on the OS side, with all the Office apps being a bonus. For the tech side, it really allows me to keep up to date on the Microsoft world, and from a practicality point of view it still works because it allows me to solve all the problems I have with my rather niche situation. (I should point out you actually get ten full license for each version of each OS/application).

I found out about this from Paul Thurott’s Supersite for Windows, and his Windows Weekly podcast, and as far as I can tell, there’s no real downside – even if I don’t continue my subscription (which drops to 150 USD in future years), the software continues to work, I just can’t re-download the .iso files.

Hope this is of value to others.

New graphics card – ATI HD 5750

Hey, don’t fall asleep!

Minor update to my setup – I added an ATI 5750 graphics card – cost about 11,000yen – basically to just to speed up WIndows 7 and play games, even old ones, at 1920*1080 like Darwinia and Far Cry with all the effects on – the old 4350 HTPC card really couldn’t do that. I got the Kuroutoshikou card with an Arctic Cooling fan on it. Very quiet and great results.

Linux / Windows

CPU: Athlon II 250 , 4GB DDR3 RAM, ASRock M3A785GMH mobo, Kuroutoshikou ATI HD 5750, Enermax fans, and Antec Earthwatts 430 PSU. 320GB Seagate HDD + 2 * 1TB Western Digital Green eSATA drives for archives. OS: Fedora 13 (64bit).

Power Reduction

I thought I’d add a minor update for my Linux ‘Typhoon‘ server regarding power consumption, given that reducing it’s noise, power and also increasing storage space was the goal of the recent updates.

Here are my wholly unscientific findings, based on my Watt checker over a couple of hours.

Functional rest at the desktop:  was ~80 Watts, down to ~60W (-25%)

Being used normally  on desktop: was ~90-100 Watts, down to ~68W (-32%)

Peaks during high CPU/HDD use: was ~115 Watts, down to ~75W (-35%)

I think that’s a pretty successful outcome to say it is quieter and such. It’s still much, much lower powered than I thought, and I’m sure the ‘new’ HDD and the ATI card are mainly responsible for that power drop.

"Typhoon" Upgrade Completed

Just to explain the title, ‘Typhoon’ is my old Linux box. After a while of little interference save the recent fan and GPU replacement, it’s been happily churning away since April 08 when I rebuilt it to merge my Windows and [even] older Linux box.

The reason for this upgrade was simply that the two 320GB drives which act as my rsync’d archive drives have essentially become full and thus in need of swapout. After a bit of looking around and review reading, I went for two 1TB Western Digital Green drives. I used to only use Seagate, but over the last few years I seem to have drifted to WD. These are 32MB cache and 5400rpm drives – I didn’t need more speed, just power efficiency and capacity. Thus far I’m happy with both.

Essentially then, one of the 320GB is now wrapped in a safe place just in case (TM) and all that data is now on the two terabyte drives. The other 320GB drive has replaced the 160GB drive as the home of my GNU/Linux OS itself. On that subject I took the opportunity to upgrade from Xubuntu 8.04LTS to 9.04. there’s another one out next month, but it’s somewhat nice to know this version has been hammered a while.

It’s been running for a few hours and passed all the tests I’ve thrown at it, so I’d say that was mission accomplished.

Big Box Update

Just a minor, certainly insignificant update to the last post here. After a bit of research, and after checking the wallet more than once, I went for a pair of Enermax Cluster fans, and a Sapphire HD 4350, passively cooled, which also has HDMI on board.

For what it’s worth, the fans also have PWM, which is the fan speed control standard, so that’s likely helping some, but overall the machine seems much quieter. The new graphics card seems nice – it’s definitely a good media playing card, but it’s also half height, so with that large 6800GS out of there, there seems to be a lot more space in the case, and since the HD 4350 doesn’t require the extra power plug, it seems I can clear a lot of cabling out of the way.

The white LEDs spinning were a lot of fun for a while, but since it generated light equivalent to the average street light, I hit the helpfully located off switch on those, and with live with a darker, quieter box.

The Big Box Dilemma

Like most people, geeks, freaks, I have more than one computer. I don’t want to go into that right now. it’d be a longer post.  Over the last couple of years I’ve thinned the herd such that I now have only one ‘big box’. As I write, the details are here, but for the sake of click saving, it’s roughly this: Athlon64 3700, DFI gaming mobo, 2GB RAM, 256MB nVidia6800GS.  I built it in December 2005 as an acceptable games rig. In the end, it got older, as did I, and thanks to family, an XBOX 360 and a PS3, I don’t really game on the PC at all now. About 18 months ago, this became my Linux server with a plug-in eSATA Windows drive for gaming. At some point in the meantime, I’ve lost that drive. The issue with home servers is that I need it quieter, and to use as little power as possible when it’s on (not 24/7), thus I did some quick power checks:

Functional rest at the desktop: ~80 Watts

Being used normally  on desktop: ~90-100 Watts

Peaks during high CPU/HDD use: ~115Watts.

This is all a lot lower than I expected, but it is still a little loud. The are two main fans which make the noise – a 120mm front fan and a 100mm rear fan, with the graphics card a little behind them.

I was thinking of getting an EeeBox or similar, but honestly, I think I can cure the noise for less than 10,000yen:

1) Replace the fans with new, slower turning quieter fans for about 6,000yen.

2) Replace the 6800GS with it’s Arctic Cooling fan with a passively cooled card for ~ 4,000yen.

I’ve already played with OS and BIOS settings, so all things being equal, I’m hoping this will make for a quieter, lower power server.

(Caveat: I know spending money on a 3.5yr machine is odd, but I think it does have a few years left. Also, the 6800GS is quite big – the newer card will be smaller, and thus maybe help heatflow also. It’s 25% the price of an Eeebox.)