Microgrants/32GB usb stick for Commons dump

Overview

My bot, Faebot has been a busy bot lately. As a result my networked hard disk that holds my local dump of commonswiki.xml gets accessed constantly from several threads running on my macmini. I would like to move the 22GB file to flash memory on USB stick to avoid problems with wear and tear on my hardware. At the moment, the largest USB stick I have to hand is 8GB but the prices of 32GB have fallen dramatically, so this seems realistic.

Bot owners are expected to use a local dump of commonswiki.xml for lengthy or complex bot work, to avoid putting a lot of transactions on the Wikimedia servers. Naturally this means that the stress of high volume transactions moves to your own home kit!

Budget

£16 for a small form-factor 32GB USB stick. See example supplier.

Timeline

• Indefinitely. I would be unable to loan it, or use in on other machines as it would be in constant bot use from my home macmini. If I stop running Wikimedia scripts that use such a dump, or the Commons dump gets too large for a 32GB stick, I would be happy to lend it on to another volunteer that has a use for it.
• Progress will be seen on Commons:User:Faebot.
• I would expect this 32GB of storage to be good at least throughout 2013. At the moment a reasonable xml dump of Commons image pages is running at 22GB, it may well exceed 32GB by 2014.

Expected outcomes

• Reduce wear and tear on my home desktop drive (putting my personal backup and archives at risk).
• Enable Faebot to continue with the 2 million+ UK Geograph image categorization plus the other odd tasks it gets up to within scope. See Commons:User:Faebot/Geograph for current projects including sorting UK images by County/Borough using Open Street Map data (probably a year of slow bot work).

Who I am

I am Fæ, I do a lot on Commons.
--Fæ (talk) 19:38, 22 October 2012 (UTC)

Discussion