Early toolbox and underlay demonstration

Toolboxes: In its infant stage, toolboxes provide macro collections for applications.  The first implementation makes its debut in DNS Manager giving clients the capability to reset and manipulate MX settings.  MX records provide control of e-mail routing with one-click settings to assign e-mail to a hosted GMail account, clone custom MX settings (before changing nameservers to ns1/2.apisnetworks.com), and reset  DNS settings if necessary.  Macros are executed server-side over AJAX, but can easily be integrated client-side with the new JavaScript API accessor.

A simple slide-down dialog in MDM reminding you to click it and don't file a ticket.

Dialogs: Two new dialogs have been added as less disruptive (and obstructive) variations of the 100%-opaque-can’t-see-a-damn-thing-beneath-the-300px-x-100px-modal dialog.  Popins and slide-downs.  Simple names, simple ideas, simple approach.  Popins reduce the opaque background to a 10 pixel border and slide-downs slide into view from the top of the screen.

Bindless-proxies: every API called during a form submission is handled by the application error handler - the dialog that slides down and fades background colors.  Yes, that annoying feature.  This behavior was implemented as a quick fix for the poor error handling in earlier versions of the control panel (early as in more than a year old), which has become more problematic as the control panel evolves.  Commands that return bools - user exists, e-mail exists, domain exists, etc, require wrappers to prevent the error handler from storing the value in the error buffer, which transforms innocuous code into spaghetti without meatballs, but with macros.  Lots and lots of macros.  Bindless proxies call the commands and mute the response without the unsightly no_warn()/no_error()/resume_warn()/resume_error()  macros.

• Added: Toolbox concept, a collection of task-specific macros per app.  The macro set for DNS Manager features hosted GMail MX provisioning, DNS reset, and MX cloning (apnscp)
• Added: Bind-free proxy- afi liaison that processes commands without triggering an error/warning during postback - atomic process of do_bind()/unbind() macros (Page Controller)
• Added: two new less obtrusive modal dialog forms- slide-ins from the top and modal dialogs will transparent underlay
• Added: apnscp.cmd()- AJAX API dispatcher
• Added: create new folders from within modal browser (Modal File Browser)
• Fixed: IE6- active folder lacks background style (Modal File Browser)
• Fixed: all except IE6- successive progress updates create memory leak during upload progress.  Remove iterative function approach by animating background position.  Works in all browsers except IE6 (File Manager)
• Fixed: omitting permitted exit codes and including process options results in options interpreted as permitted exit codes (Process)
• Fixed: inline file browser missing icons in IE6
• Fixed: attempting to overwrite an e-mail address results in a duplicate error message (Email)
• Fixed: performing an edit, revert, then edit operation in succession resulted in empty destination field (Manage Mailboxes)
• Changed: reuse modal file browser
• Changed: warn user to activate changes (Multiple Domain Manager)
• Changed: update file uploader style
• Changed: vertically align modal dialog content
• Changed: display selected file size/count in summary footer (File Manager)
• Changed: find_quota_files()- include file size on disk (User)
• Changed: use packed JavaScript and CSS where possible.  Packing occurs nightly on the client following an applicable file update
• Changed: rewrote jTip, updated style properties (Tooltip)
• Changed: warn client on ineffective DNS modification if domain is using foreign nameservers (DNS Manager)
• Changed: mute error reports stemming from nightly backend service refresh (Error Reporter)
• Changed: add_alias(), modify_mailbox()- enforce consistent alias formatting (Email)
• Changed: format forwarded e-mail addresses (Manage Mailboxes)
• Changed: split Mailbox Routes into add/edit components
• Changed: use exponentially diminishing weighted averages for quota overage forecasting, add daily growth rate statistic (Disk Usage Watcher)
• Changed: add placeholder variables to Vacation Auto-Responder documentation
• Changed: permit an infinite number of data points on a graph legend (pChart)
• Removed: htaccess Ensim application, decouple Ensim authentication from login (Auth)

• Added: control panel, WebDisk, and SOAP bandwidth logging
• Added: service grouping by HTTP host (Bandwidth Breakdown)
• Added: formatted regular expressions with variable placeholders and backreferences (REGEX)
• Added: additional PEAR channel support- list_pear_channels()/add_pear_channel(), API-only (Web)
• Added: DNS resolver with independent timeout values used to expeditiously resolve IP addresses or fail without relying on a lengthy gethostbyaddr() OS timeout (Login History)
• Fixed: teed output initialized by backend process with a persistentlock retained root ownership (Process)
• Fixed: trac_installed()- regression: /usr/local/share/trac is not automatically populated on install (Verco)
• Fixed: “Show Entries” broken on IE6 (Account Summary)
• Fixed: get_debug_bt()- stack trace misformatted multi-dimensional arrays (Error Reporter)
• Fixed: apppend_msg()- error string formatted regardless of format arguments (Error Reporter)
• Fixed: LF control character not interpreted as line-break within IE (AJAX Engine)
• Fixed: directory contents are duplicated to all siblings if the file browser automatically opens a directory that is one level below the root (File Browser
• Fixed: modal overlay height in IE6
• Changed: series of deprecated methods for MySQL privileges- add_mysql_user_permissions() -> set_mysql_privileges(), delete_mysql_user_permissions() -> revoke_from_mysql_db(), get_mysql_user_permissions() -> get_mysql_privileges() (SQL)
• Changed: detection of aborted connections while performing a file output tail - processes are reaped and the output file is truncated.  Experimental and presently available on the Traceroute application.
• Changed: add_user()- force lowercase username (User)
• Changed: MySQL Manager UI changes, introduce watered-down privilege system - read/write
• Changed: consolidate “select all” feature to apnscp JS library
• Changed: install_gem()- skip rdoc installation (Web)
• Changed: jazz up the interface for Firefox/Webkit users through rounded borders
• Changed: save user password in user options file (~/.my.cnf) on change.  This feature allows the user to use mysql from the commandline without entering the password every time.  Feature introduces set_mysql_option() and get_mysql_option() methods (SQL)
• Changed: add browser and JavaScript status css classes to body
• Changed: send the auth module used to service the request to CP bandwidth usage log (Auth)
• Changed: provide a benign explanation in the event of an interrupted backend request if the server is performing a nightly code update at 3 AM (DataStream)
• Changed: HTML clean-up, desaturated the veritable cornucopia of colors (UI)
• Changed: attach size of all files within the directory, disk space free, and limit to bottom of directory output (File Manager)
• Removed: colorful buttons
• Removed: process stderr sent as warning, this was too ambiguous (Process)

In less exciting news, MySQL has been upgraded to 5.0.81 and Postfix from 2.5 to the new 2.6 branch.  Both are routine maintenance upgrades.

- Matt

• Added: input text hints (Multiple Domain Manager, Add Subdomain, File Manager, DNS Manager)
• Added: HINFO resource record support (DNS)
• Added: interactive shell based loosely upon PHP_Shell (PEAR) + CLI authentication module to facilitate development and provide a driver for unit testing (CLI)
• Added: Inline file browser “Change Directory” accessible through “Browse…” (File Manager)
• Fixed: WebDAV delegation prevents original uid from writing to directory (File)
• Fixed: connection error to primary domain lookup server triggers fatal() (DNS)
• Fixed: named backreferences treated as separate conversion specifiers while parsing command and args (Process)
• Fixed: multiple DAV improvements- pass updated class 1 Litmus test, collection compliance, error on bad content-type header, escape all URL entities in filenames, streamline current class 1 methods - ~ 40% improvement over last benchmark (75 seconds vs 45 seconds per 500 requests) (DAV)
• Fixed: local file names with non-alphanumeric characters fail during “Download and Extract” operation in File Manager
• Fixed: disk quota value of ‘none’ interpreted verbatim (Manage Users, Add User)
• Changed: respond with error if user is given e-mail privilege, but no domains are selected (Add User)
• Changed: add package URL, minor CSS/JS changes (Package Manager)
• Changed: remove disruptive fatal() macro on MySQL query errors (MySQL)
• Changed: cache stat() responses - ~25% increase in file_stat() performance (File)
• Changed: move highlight, rename, file browser operations to global script
• Changed: cache previously expanded directories (Inline File Browser)
• Changed: rewrite copy(), add fourth parameter (prune) to abort recursive copy on error (File)
• Changed: rewrote unbuffered AJAX proxy since intercepting an XMLHttpRequest in interactive mode is neither encouraged nor widely supported.  Real-time traceroute and Rails installations are cross-browser. (ajax unbuffered)
• Changed: add pale yellow glow on hover for password and text input types
• Changed: minor HTML clean-up, convert category application listing to list (Page Template)
• Changed: clean-up process output teeing code (Process)
• Changed: tag last modified timestamp to CSS/JavaScript to encourage clients to honor expiry (Page Template)
• Changed: remove select() timeout from process draining effectively promoting to blocking sockets - whether this produces disastrous process deadlocks remains to be seen (Process)
• Removed: several disused JavaScript scripts

• Added: DNS filter control (DNS Manager)
• Added: subdomain_info(), domain_info()- return information on subdomain or domain (Web)
• Fixed: delete()- glob returns pattern on no match (File)
• Fixed: missing SPF helper javascript (SPF)
• Fixed: filter ignores full match on partial match when show restriction is in use (AJAX File Browser)
• Fixed: SOAP session id is not restored on cache hit (Auth::SOAP)
• Fixed: global subdomain triggers subdomain remap on postback (bug #105) (Manage Users)
• Fixed: make_path()- symlink fixes: 1) invalid symlinks doubly-append site filesystem prefix 2) symlink pointing to existing file treated as regular file (File)
• Fixed: get_acls()- cache check fail on symlink (File)
• Fixed: create_mailing_list()- majordomo-owner alias always created (Majordomo)
• Fixed: role permission mismatch produces similar copy of insufficient privileges error (afi)
• Fixed: stat()- unaffiliated uid results in null entry for owner/group fields (File)
• Changed: add_address()/add_alias()- error if address already exists (Email)
• Changed: UI tweaks, gradually introduce new apnscp 2 input controls
• Changed: rewrite inline file browser, add support for non-linear directory traversal
• Changed: stat()- skip ACL lookup for symlink (File)
• Changed: get_acls()- satisfy return data type for non-existent files by populating ACL cache with dummy entry (File)
• Changed: hide write operations on directories that user does not have access to write files (File Manager)
• Changed: rename_subdomain()- optional third parameter to change subdomain location (Web)
• Changed: get_acl()- persist ACL cache across requests (File)
• Changed: cache make_path()/unmake_path() translations (File)
• Changed: reload module cache on permission/argument mismatch before attempting (afi)
• Changed: add sorter to subdomain table, include owner/type attributes (List Subdomains)
• Changed: introduce unhandled exception handler (Error Reporter)
• Changed: improve delete() POSIX-compliance, use directory permissions exclusively instead of file permissions for tests;  rename() to move files is deprecated (File)
• Changed: rename auth driver App to UI (Auth)
• Removed: SOAP session persistence caused by unavoidable complications with static class members (SOAP)

• Added: auto-save session if expires; resume page/form action after logging in
• Added: subdomain_exists()- query if global/local/fallthrough subdomain is configured (Web)
• Fixed: disallow updating billing information if account has expired (Change Account Billing)
• Fixed: display_application()- postback severity compared to PB_ERROR (Page Container)
• Fixed: null response printed on ajax output (Ajax)
• Changed: Rewrite of session and authentication handlers composed of three separate authentication layers (App, SOAP, DAV).  Latency improvements are as follow (throughput ratio, (rendering latency)) :
  • Login: 1.6X (38%)
  • Dashboard: 21X (95.4%)
  • App Rendering: 16X (93.8%)
  • DAV: 6X (83.3%)
  • SOAP: 21X (95.3%)
• Changed: decrease garbage collector frequency from every page to 50%
• Changed: disable always_populate_raw_post_data PHP directive
• Changed: [add,remove,modify]_record()- add .25 second sleep after DNS modifications to provide ns time to catchup (DNS)
• Changed: delete()- improve permission algorithm (File)
• Changed: improved error handling verbosity (Error Reporter)
• Changed: add login error if account is disabled (Login)
• Changed: prefer static invoice over domain name database lookups (Billing)
• Changed: cache instantiated classes for SOAP clients (SOAP)
• Changed: lazy load PostgreSQL, Process, Graph, Compression classes (apnscpFunction)
• Changed: disambiguate subdomain interface, list primary domain first (Add Subdomain)
• Changed: clean-up code, remove backend wrapper methods (File Manager, Add Subdomain, Module Skeleton, Page Container, Auth, Web, Common, etc)
• Changed: clean-up schema, fix length primary keys (soap_keys, session_information)
• Changed: cache navigation links/dashboard data (Template Engine)
• Changed: use site id while validating SOAP key (Auth)

Secondly, subdomain fallthrough support has been added for addon domains.  Select ‘*’ for the subdomain in “Add Subdomain” to add a fallthrough for the named domain.  Fallthroughs are used if the named subdomain is not found.  If a fallthrough subdomain is not set, then the global subdomain is used, if found, otherwise a 404 is returned.

Finally, vacation messages have been added for the primary user.  Visit E-mail -> Vacation Messages to set.

Apache mapping changes require modification to account configurations.  Changes are scheduled to be made at 1 AM EDT (-0400 GMT) tongiht.  Addon domains may be temporarily unavailable for a 10 minute window during this time.

• Added: list_domains()- returns a list of domains and corresponding document roots (Web)
• Added: abort transfer feature for file uploads (File Manager)
• Added: vacation message support for the primary user (E-mail -> Vacation Message)
• Added: fallthrough subdomain support  for addon domains
• Fixed: regression in vacation message controller generates failed postback if vacation not set (Vacation Message)
• Fixed: miscellaneous code clean-up (File Manager)
• Fixed: remove spurious resource closure (Curl Layer, DataStream)
• Fixed: application dropdown fails to load script (webDisk)
• Fixed: find_quota_files()- double-escaped metacharacters (User)
• Fixed: get_quota()- match grace column hh:mm format (User)
• Fixed: missing toolbox in empty directories (File Manager)
• Fixed: regression on display body if warnings exist (Urchin)
• Fixed: negative z-index on header/menu renders behind body in Firefox 2 (Page Template)
• Fixed: add_user()- invalid method signature (User)
• Fixed: parse_ini_file()- skip conversion if empty string (Common)
• Fixed: “cannot access directory” message on empty directory (File Manager)
• Fixed: postback calls filter if filter string is empty (Manage Users)
• Changed: update chart to pChart (Bandwidth Breakdown)
• Changed: refactor afi, reduce set_user_parameters() calls, cache argument reflection results (~ (Module Skeleton, apnscpFunctionInterceptor)
• Changed: upgrade jQuery 1.3.2
• Changed: upgrade SimpleModal 1.2.3
• Changed: clean-up afi call routines (apnscpFunctionInterceptor)
• Changed: clean-up login process, use error()/warn() wrappers (Login)
• Changed: enhance login security by implementing random usleep() on invalid login (apnscp Session)
• Changed: turn-off APC stat checks
• Changed: introduce new domain mapping approach with O(1) lookups on addon domains, ~8% speed-up on subdomain lookups, and variable speed-up on primary domain fallthrough - O(n) -> O(1)
• Removed: blockUI replaced by SimpleModal (WebDAV, Add Subdomain, File Manager, Add User)

My observations and notes only represent a small slice of deficiencies.  I want your help in identifying current short-comings to be addressed in the Fifth Generation. The most successful, forward-thinking businesses are those that are driven by its clients.  The Fifth Generation aims to be the most ambitious iteration yet, so let’s set the bar high and give me a challenge.  Here are some brainstorming questions:

• What software versions would you like to see?  Ruby 1.9 and Python 2.6 are planned.
• Any layout changes, e.g. webmail locations, behaviors, subdomains, addon domains, etc?  Bear in mind these changes are more difficult to execute and possibly more disruptive.
• If you could change one thing about apis, what would it be?
• Ways to improve user experience?  How can we facilitate daily operations for you?  For example, is SpamAssassin inaccurate with tagging spam?  How would you like spam management change on the new family?

Questions are very open-ended, because any form of feedback is invaluable at this stage.  Brainstorming is a stage when tangents run amok and somewhere from that chaos [deceptively] good ideas emerge.   Software version requests are posted within the wiki.  Head on over to the forums and let’s hear not only what you need, but more importantly, what you want.  The Fifth Generation build is set to begin in May.

Discussion link
List of planned changes
Send me your ideas!

• Added: get_account_quota()- retrieve file/disk quota on group  (Site)
• Added: estimate apache quota contribution, accessible with get_quota(”apache”) (User, Disk Usage Breakdown)
• Added: find_quota_files()- generate list of files contributing towards quota (User, Disk Usage Breakdown)
• Added: map_{sub,}domain()- addon subdomain and domain links within the corresponding user’s home directory (Web, Aliases)
• Fixed: delete()- symlinks not removed (File)
• Fixed: format backreferences treated as separate format argument (Process)
• Fixed: erroneous account quota usage (Dashboard)
• Fixed: skip extraneous bandwidth values during bandwidth tabulation (Tabulate Bandwidth)
• Fixed: backend -> query() -> backend call mangles arguments (Module Skeleton)
• Fixed: custom FTP jail not set (FTP)
• Fixed: get_mailbox() invalid filter type (Email)
• Fixed: get_account_quota()- prefer quota limit response over account config diskquota,quota value (Site)
• Changed: generate graph colors using random HSL values to produce unsaturated colors, include rgb/hsl/hsv conversions (Color, Disk Usage Breakdown)
• Changed: pChart chart rendering (Disk Usage Breakdown)
• Changed: upgrade tablesorter to 2.0.3
• Removed: get_credit_card_information(): unnecessary join on credit card presence (Billing)