Microgrants/32GB usb stick for Commons dump: Difference between revisions
(ce) |
(c) |
||
| Line 4: | Line 4: | ||
<!-- A description of what you want the microgrant for should go here. --> | <!-- A description of what you want the microgrant for should go here. --> | ||
My bot, [[:Commons:Special:Contributions/Faebot|Faebot]] has been a busy bot lately. As a result my networked hard disk that holds my local dump of commonswiki.xml gets accessed constantly from several threads running on my macmini. I would like to move the 22GB file to flash memory on USB stick to avoid problems with wear and tear on my hardware. At the moment, the largest USB stick I have to hand is 8GB but the prices of 32GB have fallen dramatically, so this seems realistic. | My bot, [[:Commons:Special:Contributions/Faebot|Faebot]] has been a busy bot lately. As a result my networked hard disk that holds my local dump of commonswiki.xml gets accessed constantly from several threads running on my macmini. I would like to move the 22GB file to flash memory on USB stick to avoid problems with wear and tear on my hardware. At the moment, the largest USB stick I have to hand is 8GB but the prices of 32GB have fallen dramatically, so this seems realistic. | ||
Bot owners are expected to use a local dump of commonswiki.xml for lengthy or complex bot work, to avoid putting a lot of transactions on the Wikimedia servers. Naturally this means that the stress of high volume transactions moves to your own home kit! | |||
; Budget | ; Budget | ||
Revision as of 20:26, 22 October 2012
- Overview
My bot, Faebot has been a busy bot lately. As a result my networked hard disk that holds my local dump of commonswiki.xml gets accessed constantly from several threads running on my macmini. I would like to move the 22GB file to flash memory on USB stick to avoid problems with wear and tear on my hardware. At the moment, the largest USB stick I have to hand is 8GB but the prices of 32GB have fallen dramatically, so this seems realistic.
Bot owners are expected to use a local dump of commonswiki.xml for lengthy or complex bot work, to avoid putting a lot of transactions on the Wikimedia servers. Naturally this means that the stress of high volume transactions moves to your own home kit!
- Budget
£16 for a small form-factor 32GB USB stick. See example supplier.
- Timeline
- Indefinitely. I would be unable to loan it, or use in on other machines as it would be in constant bot use from my home macmini. If I stop running Wikimedia scripts that use such a dump, or the Commons dump gets too large for a 32GB stick, I would be happy to lend it on to another volunteer that has a use for it.
- Progress will be seen on Commons:User:Faebot.
- I would expect this 32GB of storage to be good at least throughout 2013. At the moment a reasonable xml dump of Commons image pages is running at 22GB, it may well exceed 32GB by 2014.
- Expected outcomes
- Reduce wear and tear on my home desktop drive (putting my personal backup and archives at risk).
- Enable Faebot to continue with the 2 million+ UK Geograph image categorization plus the other odd tasks it gets up to within scope. See Commons:User:Faebot/Geograph for current projects including sorting UK images by County/Borough using Open Street Map data (probably a year of slow bot work).
- Who I am
I am Fæ, I do a lot on Commons.
- Discussion