Gallery3:Modules:aws s3 - Gallery Codex
Personal tools

Gallery3:Modules:aws s3

From Gallery Codex

About

Synchronises Gallery 3 content with an Amazon S3 bucket for high traffic sites, backup purposes, or just for that added speed increase/load decrease on your web hosting server.

Description

This module allows you to synchronise your Gallery content with Amazon Simple Storage Service (S3) for either backup or high traffic sites. It also supports (public content only at present) the use of Amazon CloudFront (CF) and the use of DNS CNAME's as opposed to Amazon's default S3/CF domain names (i.e; http://cdn.danneh.org as opposed to http://xyzhsjksdfs.cloudfront.net or http://danneh-org.s3.amazonaws.com/).

Installation

This module may be downloaded from my website here. Note: I will no longer be updating gallery3-contrib git repository, as it's just easier for me to maintain one repository of all my code on my website.

To install, extract the "aws_s3" folder from the zip file into your Gallery 3 modules folder. Afterwards log into your Gallery web site as an administrator and activate the module in the Admin -> Modules menu. Configure the module: Settings -> Amazon S3 page. You will be required to enter your Amazon S3 settings (Access Key, Secret Key, Bucket Name). Optionally, you can change the URL template below for use with CloudFront.

Known Issues

  • Spaces (or other odd characters) in filenames cause the refusal of the upload by S3.
  • This module currently does not work with CloudFront and private/permission restricted content due to the Query String Authentication (QSA) signature differences between CF and S3.
  • There is an issue affecting the use of SSL to transfer data between G3 and S3. Suggested work-around for now is to simply disable SSL data transfers. This does not affect the use of SSL for G3 itself.
  • I'm also aware of problems when modifying images locally (i.e; rotating) that the image is not updated on S3, so it appears as though nothing has happened, even though the image has changed locally.

Please report all other issues with this module on this module's official G3 thread here.

S3 Primary Storage

As previously mentioned, I was looking into the idea of allowing G3 to be used as a gallery application, but for this module to essentially allow G3 to store all of its data on S3 alone. After much research, deliberation, and attempting to achieve this, I have determined that it is not possible with the current G3 application. There are many reasons behind this as to why I have "given up" so to speak, but the bottom line is that G3 simply does not have enough hooks to allow this module to extend its filesystem out to the cloud. There's too much going on, and it's not as extensible as I'd like it to have been in order to complete this feature. As a result, S3 primary storage will NOT be featured in v3, or any other future release of this module.

Change Log

29th Jan 2011 : Version 2.1 : Maintenance Release

  Updated
  - Implemented version numbers into 2 digits to allow for minor version releases.
  - Added extra fields to database to cache MD5 hashes of files which are used in comparison code later on.
  - Re-sync task re-visited. Removed "Emptying bucket" methods, as it no longer seemed necessary after addition of hash comparison code.
  - Resolved issues moving entire albums from one place to another.

20th Jan 2011 : Version 2 : General Update Release

  Added Features
  - Upload select types of images only (thumbs/resizes/fullsizes).
  - Schedule initial synchronisation during gallery shutdown to prevent browser hanging on large installations.
  - Schedule upload of new items after upload during gallery shutdown to prevent browser hanging on slower uplink connections (DSL home servers, etc).
  Bug Fixes
  - Added verification routine to verify entered bucket name and access credentials to avoid false hope.
  - Added check in module's installation to prevent activation should curl not be installed, as the S3 library depends on it.
  - Resolved some issues with cooliris slideshow player not working correctly, due to browsers blocking ajax calls for images since it's cross-domain. The module will now allow the browser to pull the images from the local server to prevent any xdr issues.
  - Fixed problems moving items around. The module now moves content around the s3 bucket to coincide with the local filesystem and g3's perception of items and albums.
  - Resolved problems with odd characters in filenames. This actually wasn't to do with odd characters but to do with the filename being escaped twice, causing file not found errors.

26th Nov 2010 : Version 1 : Initial Release