Releases
Roadmaps for specific releases.
Stability and Reliability
- Gallery fails hard when graceful handling of unexpected events is often possible.
- Log such events for the admin
- Try to handle the requests as gracefully as possible such that visitors don't notice such issues.
- Check the forums for frequent errors and see how we can handle them more gracefully
- Reconsider the design of long-running tasks / batch operations.
- Maybe based on the new asynchronous job handler (scheduler), we can improve the reliabilty of long-running tasks. E.g. split the task into managable subtasks and have Gallery guarantee that all of them get executed or logged.
- Consider dropping non-transactional DB support
Improve Reliability of Installer / Upgrader Authentication
Consider Improving or Dropping flock-based Locking
- Flock generates a lot more support requests and errors than Database-based locking.
- Many locking support issues can be resolved by switching to DB locking. This could imply an inherent unreliability of flock or the presence of bugs in the G2 FlockLockSystem.
Data Integrity
The framework should make it hard to violate data integrity and there's much room for improvement.
- Filesystem operations are not part of the transactions yet!
- An expensive way to deal with that is to have a DB transaction around each FS operation.
- Consider two-phase commits (see task on sf.net)
- Make certain filesystem operations transactional (create, move, delete) coupled to the DB transactions
- There are no foreign keys in the database yet.
- There are no enums yet (e.g. for the entityType)
- There are quite a few unstructured data fields in the database (toolkit operations, theme data / settings, ...) which make it hard to enforce integrity or make it expensive to update / delete data e.g. when a module is upgraded / uninstalled / missing.
- Provide an integrity check / repair module
- On platform errors, try to chmod and try again before failing hard
Usability
See: Usability Issues
- Site admin redesign
- Item admin redesign
Security
There are certain security improvements that should be implemented to make the framework more robust. Core developers can contact valiant to get the internal security status report.
Load, Performance and Scalability
Test Framework and Usage Heuristics
Before we refactor major components (MPTT, permissions / ACL, ...), we need to acquire some base data first.
Please see: Development Document about Load, Performance and Scalability.
Minimize I/O
As a simple rule to minize the response time and load on the server, we should minimize the Input/Output:
- Minimize client-server I/O: Page size / HTML output (e.g. through AJAX, minimizing rating interface, ...)
- Examine all Javascript, rating interface
- Minimize filesystem I/O:
- less PHP files to include, smaller PHP files to include in a default request. Reduce # of class files required for an individual page load.
- less g2data/ reads per request (grouping cached data together, based on how often data changes)
- cache event handlers and other module data -> don't include all module files on each request
- Refactor locale files (move into central folder structure)
- Add Memcached (/APC/eAccelerator) as alternative to Disk/DB-Cache
- Minimize database I/O:
- Group data together, e.g. {user, session}, ...
- Optimize caching
- Locate, Analyze and Optimize db queries using most CPU time
Disable Disk Caching
- Some disk caching has definitely a detrimental effect on performance. E.g. when loading a batch of 50 entities a DB query is sure faster than 50 file reads.
- Other disk caching might still be worthwhile. E.g. a single cache file for most static config data that is required on each request, parsing all active module.inc files into a single aggregate file, etc.
Refactor Permission / ACL Design
- It scales very badly with the number of user-albums / with heterogenous permissions
Refactor Item Hierarchy
- MPTT looks promising and could make a lot of frequent queries a lot faster
Refactor Slideshow Module
See: http://tieguy.org/blog/2007/01/30/performance-tip-for-gallery-2-search-engine-bots/
The underlying problem is that the slideshow module scales very badly. We need to refactor its design. And since users aren't happy with the look and the lacking configurability, we can basically replace the whole module.
Food for thought: A RSS feed which allows to specify a range (like SQL's LIMIT) could feed an AJAX based slideshow that requests the next 20 image URLs / descriptions every once in a while.
The same RSS feed could be used for external slideshows (embedding a small slideshow on other websites is what people do and like about flickr, ...).
Related feature requests / tasks / bugs:
1144960, 1096995, 86440, 1309306, 1564714, 1390690, 1081693
Slow DB Queries
- Identify slow DB queries and queries that don't scale well.
Seamless and Easy Integration
See: Integration / Embedding Roadmap
Testing and QA
- Improve the unit-test framework
- Add tests for views
- Improve the testability by converting the core API from static function calls to instance method calls
- Automate testing with continuous build(s)
- Automate testing across platforms (e.g. via appliances / testing server farm)
Maintainability
Code smells:
- Different image classes GalleryPhotoItem, GalleryDerivative, WatermarkImage, ThumbnailImage. Other GalleryDataItem are very similar.
- What they have in common: fetchPath() and getMimeType().
- Either introduce an interface and/or introduce a new class that covers common image aspects.
- The permission / ACL system acts on itemIds, but WatermarkImage and ThumbnailImage (both no GalleryItem instances) are using permissions as well.
- The class hierarchy is pretty deep (Entity > ChildEntity > FileSystemEntity > Item > DataItem > PhotoItem).
- The webserver/PHP stack handles flat hierarchies better (e.g. less files to include) and with our DB schema, it would be less tables to join.
- Inheritance is limiting. E.g. An Item must be a filesystem entity and it can only have a single filesystem entity.
- (Too many) code changes necessary to associate an existing Item-type with new mime-type / renderer.
- AJAX handlers are "immediate, controller-like" views.
- Some views do more than just display stuff (e.g. ItemAddFromWeb).
Introduce Interfaces
- Need to assess what interfaces make sense (e.g. GalleryItem, Downloadable)
Refactor Entity Hierarchy
- Still very vague, but the hierarchy should be flattened. Dependencies should be removed by switching from inheritance to composition for some aspects.
PHP Version Compatibility
- Break PHP 4 compatibility with G2.4 (first release in 2008)
- Add PHP 6 compatibility with G2.4
- PHP 4 and PHP 6 support is mutually exclusive:
- var vs. class member visibility keywords (public, private, ...)
- Add function_exists('get_magic_quotes_gpc')
- Add "static" keyword where necessary
- Enable G2's PHP error handler by default
- Stop using return by reference
- Use PHP 5+ features (exceptions, interfaces, ...)
- Maybe use PHP 5.2's filter extension
Features
Features that are either essential for a mature product or features that require core module / framework changes.
Additionally, consider implementing popular feature requests: http://galleryproject.org/sfvote
Global Status Messages
Need a way for modules to easily add content to <head>/<body> header/<body> footer
- Needs a major theme API change to ensure that all themes show that status message(s) / warnings
Audit trail
- Log all events / errors / ... Similar to drupal
- Add versioning to all entities
Scheduler
An asynchronous job handler is required for a lot of features. See this forum discussion
- Cleaning watchdog table
- Cleaning sessions
- Cleaning cache
- Notifications (daily, weekly, monthly)
- Expiring / Unpublish content
- Publish content / Notify author of content that is scheduled to be published in 1 day
- Different scheduler trigger engines: cron, autorefreshing page, low-CPU check on every page impression, ...
- Contact ckdake about "expire" module by the Department of State and see sfvote to update dependent RFEs when the scheduler is available
Watchdog
We need a way to be able to track errors that are happening in g2, that aren't user visible so that an admin can get a reading on the health of their site. Example usages:
- Images that fail to build
- Failed login attempts
- any errors that bubble up to main.php
Remove NavId / StatusId
- Get rid of the g2_statusId and g2_navId from the urls. If this means deleting those two features, I'm totally ok with it. They're marginal at best (especially statusId) but they clutter up our urls.
Move Utilities into Core API
- Move GalleryUtilities into the GalleryCoreApi, but allow the old code to delegate back. (this might not be API compatible-- requires some thought)
Emergency tools
- integrity check
- repair tools
More Permission Bits
- Extend the allowable number of permissions from 31 to 256. (Hopefully this will be API compatible). (User:talmdal)
Categories / relational organization
- g2 should support relational and hierarchical organization
AJAXify the Progress Bar
- Could be a popup / overlay instead of an interstitial page
Allow multiple files per item (multiple "Views" / perspectives)
- application: multiple images can represent the same item (e.g. front / back / side view)
- decouple "item" from filesystemEntity (composition instead of inheritance)
Modular view / download permissions
- a module might want to control access based image dimensions
- a module might want to control access based on geolocation
API breaking changes we could make
- Move color packs down into themes, so each theme has its own set of color packs
- Move GalleryUtilities into the GalleryCoreApi
- Make GalleryCoreApi non-static
- Unify rendering to new getRenderer / renderer pattern
- Add return status to canBeViewedInline and less hardcoding in there (to make it a lot easier to extend existing item types to new mime-types and customization)!
- Change the handling of GalleryStatus code to support warnings. Maybe something more OO like typed exceptions extending a base class.
- Register event listeners in the database, like registered factories, instead of loading all modules on each request. http://thread.gmane.org/gmane.comp.web.gallery.devel/1854/focus=2090
- An object's "entity type" should never disagree with it's class name, otherwise when the object is unserialized, the framework will import the class file corresponding to the entity type and unserialize an object whose class is not available. This causes PHP to halt with a fatal error.
- Drop the GalleryEntity::setEntityType setter. If it's ever called, Gallery will probably eventually have a fatal error, for instance if the object is cached.
- Drop the GalleryEntity::getEntityType getter. Use GalleryEntity::getClassName instead.
- Rename the "entityType" member "className" and ensure it always agrees with the value returned by GalleryEntity::getClassName.
These changes would eliminate any possible discrepancy between "entity type" and class name by dropping the concept of "entity type" in favor of class name.
Modules to improve
- Rewrite: The current model of centralizing the rule management in a single module is flawed. We wind up causing the module developer to write complex rules that the rewrite module can parse in order to manage the rewrite rules on our behalf. It would be far easier to settle on having the rewrite module insert one rule into .htaccess that parses the entire short url and then dispatch it to the individual module in a way that it can easily parse and use. The module generated the url, it can parse it just as easily. There's no need to try to codify everything in such a way that the rewrite rule can understand it.
- RSS: This is way, way, way too complicated. It needs to be much easier to install and setup for the common case.
Modules to finish and include
- XMLRPC (needs update, refactoring, review, and probably a bit more)
- cmmigrate (needs review)
- embed (needs update and refactoring)
- Notification (Needs major rewrite)
Bugfixes
Fixing high priority bugs is an ongoing task. See http://sourceforge.net/tracker/?group_id=7130&atid=107130
