
/*

  SmartClient Ajax RIA system
  Version v9.0p_2021-05-01/LGPL Deployment (2021-05-01)

  Copyright 2000 and beyond Isomorphic Software, Inc. All rights reserved.
  "SmartClient" is a trademark of Isomorphic Software, Inc.

  LICENSE NOTICE
     INSTALLATION OR USE OF THIS SOFTWARE INDICATES YOUR ACCEPTANCE OF
     ISOMORPHIC SOFTWARE LICENSE TERMS. If you have received this file
     without an accompanying Isomorphic Software license file, please
     contact licensing@isomorphic.com for details. Unauthorized copying and
     use of this software is a violation of international copyright law.

  DEVELOPMENT ONLY - DO NOT DEPLOY
     This software is provided for evaluation, training, and development
     purposes only. It may include supplementary components that are not
     licensed for deployment. The separate DEPLOY package for this release
     contains SmartClient components that are licensed for deployment.

  PROPRIETARY & PROTECTED MATERIAL
     This software contains proprietary materials that are protected by
     contract and intellectual property law. You are expressly prohibited
     from attempting to reverse engineer this software or modify this
     software for human readability.

  CONTACT ISOMORPHIC
     For more information regarding license rights and restrictions, or to
     report possible license violations, please contact Isomorphic Software
     by email (licensing@isomorphic.com) or web (www.isomorphic.com).

*/
//> @class DataSource
// A DataSource is data-provider-independent description of a set of objects
// that will be loaded, edited and saved within the user interface of your application.
// <P>
// Each DataSource consists of a list of +link{dataSource.fields,fields} that make up a
// DataSource <code>record</code>, along with +link{dataSourceField.type,field types}, 
// +link{dataSourceField.validators,validation rules}, 
// +link{dataSourceField.foreignKey,relationships} to other DataSources, and other
// metadata.
// <P>
// The abstract object description provided by a DataSource is easily mapped to a variety of
// backend object models and storage schemes.  The following table shows analogous terminology
// across systems.
// <table border=1 class="normal">
// <tr>
//   <td>Isomorphic SmartClient</td>
//   <td>Relational Database</td>
//   <td>Enterprise Java Beans (EJB)</td>
//   <td>Entity/Relationship Modeling</td>
//   <td>OO/UML</td>
//   <td>XML Schema/WSDL</td>
//   <td>LDAP</td>
// </tr><tr>
//   <td>DataSource</td>
//   <td>Table</td>
//   <td>EJB class</td>
//   <td>Entity</td>
//   <td>Class</td>
//   <td>Element Schema (ComplexType)</td>
//   <td>Objectclass</td>
// </tr><tr>
//   <td>Record</td>
//   <td>Row</td>
//   <td>EJB instance</td>
//   <td>Entity instance</td>
//   <td>Class instance/Object</td>
//   <td>Element instance (ComplexType)</td>
//   <td>Entry</td>
// </tr><tr>
//   <td>Field</td>
//   <td>Column</td>
//   <td>Property</td>
//   <td>Attribute</td>
//   <td>Property/Attribute</td>
//   <td>Attribute or Element (SimpleType)</td>
//   <td>Attribute</td>
// </tr></table>
// <P>
// DataSources can be +link{group:dataSourceDeclaration,declared} in either JavaScript or XML
// format, and can also be +link{group:metadataImport,imported} from existing metadata formats,
// including XML Schema.
// <P>
// <i>Data Binding</i> is the process by which
// +link{dataBoundComponent,Data Binding-capable UI components} can automatically configure
// themselves for viewing, editing and saving data described by DataSources.  DataBinding is
// covered in the +docTreeLink{QuickStartGuide,QuickStart Guide}, Chapter 6, <i>Data
// Binding</i>.
// <P>
// +link{group:clientServerIntegration,Data Integration} is the process by which a DataSource
// can be connected to server systems such as SQL DataBases, Java Object models, WSDL web
// services and other data providers.  Data Integration comes in two variants: client-side and
// server-side.  +link{group:serverDataIntegration,Server-side integration} uses the
// SmartClient Java-based server to connect to data represented by Java Objects or
// JDBC-accessible databases.  +link{group:clientDataIntegration,Client-side integration}
// connects SmartClient DataSources to XML, JSON or other formats accessible via HTTP.
// <P>
// DataSources have a concept of +link{group:dataSourceOperations,4 core operations} ("fetch",
// "add", "update" and "remove") that can be performed on the set of objects represented by a
// DataSource.  Once a DataSource has been integrated with your data store, databinding-capable 
// UI components can leverage the 4 core DataSource operations to provide many complete user
// interactions without the need to configure how each individual component loads and saves
// data.
// <P>
// These interactions include +link{listGrid,grid views}, +link{TreeGrid,tree views}, 
// +link{detailViewer,detail views}, +link{DynamicForm,form}-based 
// +link{dynamicForm.editRecord,editing} and +link{dynamicForm.saveData,saving},
// grid-based +link{listGrid.canEdit,editing} and +link{listGrid.saveByCell,saving},
// and custom interactions provided by +explorerExample{patternReuse} custom
// databinding-capable components.
// 
// @see interface:DataBoundComponent for information on DataBound Components
// @see group:dataSourceDeclaration for how to create DataSources
// @treeLocation Client Reference/Data Binding
// @serverDS allowed
// @visibility external
//<
isc.defineClass("DataSource");

//> @type DSOperationType
// One of the four basic operations that can be performed on DataSource data: "fetch",
// "add", "update", "remove".  Elsewhere called CRUD operations, where CRUD stands for
// "create", "retrieve", "update", "delete", which correspond to "add", "fetch", "update" and
// "remove" in SmartClient terminology.  See +link{group:dataSourceOperations} for a full
// description.
// <p>
// There are also additional, non-CRUD operations explained below.
// </ul>
//
// @value "fetch"        Fetch one or more records that match a set of search criteria.
// @value "add"          Store new records
// @value "update"       Update an existing record
// @value "remove"       Remove (delete) an existing record
// @value "custom"       perform some arbitrary custom logic that is not a CRUD operation.
//                       Format of the inputs and outputs is unconstrained, and the operation
//                       will be ignored for cache sync purposes by +link{ResultSet}s.  See 
//                       +link{DataSource.performCustomOperation()}.
// @value "validate"     Run server-side validation for "add" or "update" without actually
//                       adding or updating anything.  See +link{DataSource.validateData()}.
// @value "viewFile"     Retrieve a file stored in a binary field in a DataSource record, and
//                       allow the browser to choose whether to view it directly or prompt the
//                       user to save.  See +link{group:binaryFields}. 
// @value "downloadFile" Like "viewFile", but the HTTP header Content-Disposition is used to
//                       suggest that the browser show a save dialog.  See +link{group:binaryFields}. 
// @value "clientExport" Upload formatted client data and export it to Excel, XML and other 
//                       formats.  Used automatically by 
//                       +link{method:dataSource.exportClientData(),exportClientData()} 
//                       and cannot be used directly.  Usable only with the SmartClient server
//                       framework.
// @visibility external
//<

//> @object Record
// A Record is an ordinary JavaScript Object with properties that are treated as data to
// be displayed and edited by a +link{DataBoundComponent}. 
// <P>
// +link{DataBoundComponent}s have a concept of +link{dataBoundComponent.fields,named fields},
// where values for each field are found under the same-named property in a Record.
// <P>
// A Record is always an ordinary JavaScript Object regardless of how the record is 
// loaded (static data, java server, XML web service, etc), and so supports the
// normal behaviors of JavaScript Objects, including accessing and assigning to properties
// via dot notation:
// <pre>
//     var fieldValue = record.<i>fieldName</i>;
//     record.<i>fieldName</i> = newValue;
// </pre>
// <P>
// The concept of working with Records is common to all +link{DataBoundComponent}s, although
// individual DataBoundComponents may work with singular records (+link{DynamicForm}) or
// may work with lists (+link{ListGrid}), trees (+link{TreeGrid}), or cubes
// (+link{CubeGrid}) of records.
// <P>
// Individual DataComponents may also look for special properties on Records which
// control styling or behavior for those records, such as
// +link{listGrid.recordEditProperty,<code>record.canEdit</code>}.
// 
// @treeLocation Client Reference/Data Binding/DataSource
// @visibility external
//<

//> @groupDef dataSourceDeclaration
// DataSources can be specified in XML format, in which case the SmartClient server is used to
// load the DataSource, or DataSources can be programmatically created on the client.
// <P>
// Whether a DataSource is loaded via the SmartClient server or programmatically created
// client-side, identical requests will ultimately be submitted to the server.  However,
// DataSources defined in XML are loaded and used by the SmartClient Server, enabling many
// features including synchronized client-server validation, request bundling, file upload, and
// optional automatic SQL/JPA/Hibernate connectivity (see the 
// +link{group:iscServer,Server Summary} for details).
// <P>
// <var class="smartclient">
// DataSources created on the client use the same style of creation as DataBound components:
// <pre>
//    isc.DataSource.create({
//        ID:"supplyItem",
//        fields:[
//            {name:"itemName", ... }
//            ...
//        ]
//    });
// </pre>
// </var>
// <var class="smartgwt">
// DataSources can be programmatically created on the client like so:
// <pre>
//    DataSource ds = new DataSource();
//    ds.setID("supplyItem");
//    DataSourceTextField nameField = new DataSourceTextField("itemName", "Name");
//    // create other fields
//    ds.setFields(nameField, ...);
// </pre>
// </var>
// Reference for all properties that can be set for DataSources, their fields and validators is
// given in the +link{class:DataSource} class reference.
// <P>
// DataSources defined in XML declare fields, validators and other settings using XML tags:
// <pre>
//     &lt;DataSource ID="supplyItem"&gt;
//         &lt;fields&gt;
//             &lt;field name="itemName" type="text" title="Item"/&gt;
//             &lt;field name="SKU"      type="text" title="SKU"&gt;
//                 &lt;validators&gt;
//                     &lt;validator type="integerRange" ... /&gt;
//                 &lt;/validators&gt;
//             &lt;/field&gt;
//         &lt;/fields&gt;
//     &lt;/DataSource&gt;
// </pre>
// DataSources defined in XML are loaded by using the <code>DataSourceLoader</code>
// servlet provided by the SmartClient Server.  This can be done as an ordinary HTML
// &lt;script&gt; tag as you application first loads:
// <pre>
//     &lt;SCRIPT SRC=isomorphic/DataSourceLoader?dataSource=supplyItem,employees,worldDS&lt;/SCRIPT&gt;
// </pre>
// .. or can be done on the fly via +link{DataSource.load()}.
// <P>
// Alternatively, in JSP environments, XML DataSources can be loaded via a special JSP tag
// supported by the SmartClient Server:
// <pre>
//     &lt;%&#64; taglib uri="/WEB-INF/iscTaglib.xml" prefix="isomorphic" %&gt;
//     ...
//     &lt;SCRIPT&gt;
//     &lt;isomorphic:loadDS ID="supplyItem"/&gt;
//     &lt;/SCRIPT&gt;
// </pre>
// <P>
// When loading an XML DataSource, by default, the ISC Server will look for a file named
// <code>&lt;dataSourceId&gt;.ds.xml</code> in the <code>/shared/ds</code> subdirectory under
// webroot.  The location of this directory can be changed, or individual DataSources can be
// placed in arbitrary locations.  For more information, see
// +link{server_properties,[webroot]/WEB-INF/classes/server.properties}.
// <P>
// XML DataSources can also be generated on the fly in case the entire DataSource or portions
// of it are based on dynamic data.  See the server API
// com.isomorphic.DataSource.addDynamicDSGenerator().
//
// @see class:DataSource
// @see group:loadDSTag
// @treeLocation Client Reference/Data Binding/DataSource
// @title Creating DataSources
// @visibility external
//<



//> @groupDef i18n
//
// <b><u>Internationalizing SmartClient Applications</u></b>
// <p>
// Internationalizing a SmartClient application involves creating locale-specific
// versions of all strings, images, and possibly colors.  In some cases, component layout may
// be affected as well (for example date field order differs between some locales).
// <p>
// <u><b>Character Encodings</b></u>
// <p>
// In order to deliver content in different languages to the browser and have it be displayed
// correctly, it is important that the browser and server agree on the character encoding being
// used to send the page.  
// <p>
// Generally speaking, you can use whatever character encoding you prefer, so long as
// you're consistent about serving your files to the browser with exactly the same encoding
// as was used to save the file to disk.  Keep in mind though that the character encoding you
// choose must be supported by all browsers you intend to support.  Isomorphic recommends that,
// where possible, you use the UTF-8 encoding.  Regardless of which character encoding you
// choose, keep in mind that for the browser to correctly display non-ASCII characters, you
// <b>must</b> explicitly set the character encoding of your page to match the encoding you
// used to save the file.  Browsers have built-in heuristics to guess the character encoding,
// but they can't be relied upon to work 100% of the time.
// <p>
// There are two ways to explicitly tell the browser what character
// encoding is being used for a given page.  One is to set the "Content-Type" HTTP header, for
// example:
// <pre>
// Content-Type: text/html; charset=UTF-8
// </pre>
// If you're using JSP on the back-end, you can set this header as follows:
// <pre>
// &lt;%@ page contentType="text/html; charset=UTF-8"%&gt;
// </pre>
// Or using Java Servlet APIs in a Servlet as follows:
// <pre>
// response.setContentType("text/html; charset=UTF-8");
// </pre>
// Note that the latter needs to be done before any content is written to the response output
// stream.
// <p>
// The other approach to setting the content encoding is to use an HTML META tag to embed it
// in the page itself (note that this applies to HTML documents only).  The META tag must go
// inside the &lt;HEAD&gt; HTML element - e.g. as follows:
// <pre>
// &lt;HEAD&gt;
// &lt;META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=UTF-8"&gt;
// &lt;/HEAD&gt;
// </pre>
// In addition, for a .html or other static (non-.jsp) file, you must also ensure that the file
// itself is saved in UTF-8 encoding.  Advanced text editors (including the Eclipse IDE) can
// show you the encoding of a text file and allow you to change it.  In Eclipse this is in the
// Preferences area (search for "Encoding") and UTF-8 is <b>not the default</b> for .html
// files, and should be changed.
// <P>
// Isomorphic recommends that you follow <b>all</b> of the above approaches for maximum
// compatibility.  For example, if you omit the META tag approach, pages saved by the user
// using the browser save function may not render correctly when subsequently read from disk
// because HTTP headers are not available in that context.  Conversely, not providing the
// HTTP header or not saving the file in UTF-8 can result in application servers delivering the
// file in the encoding in which it is saved, or in their own default and incorrect content
// encoding.
// <p>
// If you're using a given character encoding pervasively in your pages, you can also configure
// your web server or application server to use that character encoding as the default for all
// pages of a given mime type or some other criteria (depending on the capability of your
// server) - for example on a per-directory basis.
// <p>
// For more information on character encodings, character sets, and working with HTML, please
// see W3C's tutorial here:
// +externalLink{http://www.w3.org/International/tutorials/tutorial-char-enc/}
// <p>
// <b>NOTE: Default Encoding</b>
// <p>
// As mentioned above, Isomorphic recommends the use of UTF-8 encoding.  However, it is 
// not possible to configure some servlet engines to do this by default.  Both Tomcat and Jetty 
// default to the ISO-8859-1 character set (commonly known as Latin-1), even if the Java VM 
// specifies a different encoding (the Tomcat documentation claims that this behavior is a 
// requirement of the Servlet specification).
// <p>
// To work around this, we change charset encoding to UTF-8 on every request/response that 
// goes through two core Isomorphic servlets: <code>IDACall</code> and <code>DataSourceLoader</code>.
// If you need to change this, you can do so by adding <code>init-param</code> settings to your
// <code>web.xml</code> file.  Please see the Javadocs for those two servlets for examples of 
// how to do this.
// <p>
// <u><b>Framework Message Localization</b></u>
// <p>
// SmartClient components include standard prompts and error messages in various cases, and all
// such messages support localization.
// <P>
// <var class="smartclient">
// Language packs are included in the directory smartclientSDK/isomorphic/locales.  To load a
// language pack, simply include the desired locale via a
// standard HTML &lt;script src=&gt; tag, for example: 
// <pre>
//    &lt;SCRIPT SRC="isomorphic/locales/frameworkMessages_pt.properties"&gt;&lt;/SCRIPT&gt;
// </pre>
// .. to load the Portuguese locale. To see various locales, have a look at the 
// +explorerExample{changeLocales,Localization example}.
// </var>
// <var class="smartgwt">
// SmartGWT ships with pre-built language packs for many languages.  These are automatically 
// used by the framework: all you have to do is ensure that the appropriate locale has been
// set using one of the 
// <a href=https://developers.google.com/web-toolkit/doc/latest/DevGuideI18nLocale#LocaleSpecifying>standard GWT mechanisms</a>.
// In brief, either:<ul>
// <li>Add a meta tag to your bootstrap page: <code>&lt;meta name="gwt:property" content="locale=ja_JP"&gt;</code></li>
// <li>Specify the locale in a query parameter: <code>http://www.example.org/myapp.html?locale=fr_CA</code></li>
// </ul>
// </var>
// <P>
// If you find that the language pack you are using has any incorrect or missing translations,
// or you want to add a new language pack, please visit
// +externalLink{http://forums.smartclient.com/showthread.php?t=9987,this forums thread} for
// instructions on how to contribute translations so that they will be added to future
// SmartClient builds.
// <P>
// You can alternatively maintain your own private additions or overrides to the default
// language packs.
// <var class="smartclient">
// A complete list of standard messages is presented +link{group:i18nMessages, here}.<br>
// To customize these messages, use +link{Class.addProperties()} or
// +link{Class.addClassProperties()} to override default values at the instance or class level
// respectively.  For example, the class property
// +link{classAttr:RPCManager.timeoutErrorMessage} is displayed to the user in a warning dialog
// when a SmartClient operation fails to complete.  This can be modified using the
// +link{Class.addClassProperties()} method, as follows:
// <pre>
// isc.RPCManager.addClassProperties({timeoutErrorMessage:"Custom Timeout Message"});
// </pre>
// </var>
// <var class="smartgwt">
// Use the standard GWT technique for selectively overriding Messages: create a
// subclass of <code>SmartGwtMessages</code> that overrides specific methods, and a 
// .properties file with your overridden messages.  Then create your overridden Messages
// instance and call <code>I18nUtil.initMessages()</code> with it.  For example:
// <pre>
//    public class CustomizedSgwtMessages extends com.smartgwt.client.i18n.SmartGwtMessages {
//       // override button_title
//       String button_title();
//    }
//   
//    // then in a new file CustomizedSgwtMessages.properties
//    button_title=My Title
//
//    // and in a new file CustomizedSgwtMessages_fr.properties
//    button_title=French translation
//
//    // then in your onModuleLoad() call
//    CustomizedSgwtMessages myMessages = GWT.create(CustomizedSgwtMessages);
//    I18nUtil.initMessages(myMessages);
// </pre>
// </var>
// <p>
// <u><b>Application Message Localization</b></u>
// <p>
// <var class="smartclient">
// Text such as button titles typically appears in the middle of JavaScript code:
// <pre>
//     isc.Button.create({ title: "Click me!"});
// </pre>
// If you are on the Java platform and your JavaScript code appears in a .jsp, you can use the
// JSTL Core package (Java Server Pages Standard Tag Library) to replace that static "Click
// me!" text with a localized reference.  This can be done as follows:
// <pre>
//     &lt;%@ taglib prefix="fmt" uri="/WEB-INF/fmt.tld" %&gt;
//     &lt;SCRIPT&gt;
//     isc.Button.create({ title: &lt;fmt:message key="buttonTitle"&gt; });
//     &lt;/SCRIPT&gt;
// </pre>
// Assuming you've set the current Locale and provided a resource bundle with the relevant
// translated text, the above would create a button with a title that comes from the locale-specific
// bundle you provided.  For more information, see Sun's i18n tutorial:
// +externalLink{http://java.sun.com/docs/books/tutorial/i18n/} and the JSTL home page:
// +externalLink{http://java.sun.com/products/jsp/jstl/}.
// <P>
// Note that this approach does not imply that your JavaScript must appear in the midst of an
// HTML page.  A .jsp file can be used to return a pure JavaScript response via just removing
// the &lt;SCRIPT&gt; tags in the example above, and targetting the .jsp with a &lt;SCRIPT
// SRC=&gt; tag from another .jsp or .html file.
// <P>
// Approaches for non-Java platforms are similar, and generally any approach that works for
// HTML or JavaScript files works with SmartClient.  See the end of this section for pointers
// to additional resources.
// </var>
// <var class="smartgwt">
// String localization in SmartGWT is best done using <a href=http://code.google.com/webtoolkit/doc/latest/DevGuideI18n.html>
// standard GWT approaches</a>.  Although GWT supports various ways of localizing strings, 
// Isomorphic recommends the use of an approach based on Resource Bundles.  Other parts of 
// SmartGWT - for example, <code>.ds.xml</code> files - are best localized using resource 
// bundles, so using resource bundles makes it easier to share messages that are used both
// client and server side.
// </var>
// <p>
// <u><b>Support for Right-to-Left (RTL) languages</b></u>
// <P>
// SmartClient includes support for RTL languages.  To enable, simply set
// <code>dir="rtl"</code> on the HTML element:
// <pre>
//    &lt;HTML dir="rtl"&gt;
// </pre>
// ListGrid columns, horizontal layouts, scrolling directions, etc will reverse order
// automatically.
// <p>
// Because RTL in some cases requires additional media files, RTL is only supported for the
// Enterprise, EnterpriseBlue and Graphite skins.
// <p>
// <u><b>DataSource and Component XML localization</b></u>
// <p>
// Please see the separate article on 
// +link{group:dataSourceLocalization,DataSource and Component XML Localization}
// <p>
// <u><b>Localized Number and Currency Formatting</b></u>
// <p>
// Please see the separate article on 
// +link{group:localizedNumberFormatting,Localized Number Formatting}
// <p>
// <var class="smartclient">
// <u><b>Image, CSS localization</b></u>
// <p>
// Most SmartClient components use a mixture of text, CSS and images to render.  If you wish to
// provide locale-specific images or use locale-specific CSS, you can create localized copies
// of a SmartClient skin named after the locale and modify images as you see fit.  A skin is
// specified at load time using either the <code>skin</code> attribute of the
// <code>&lt;isomorphic:loadISC&gt;</code> tag or by loading the skin
// directly using a script tag like so: &lt;SCRIPT SRC=/isomorphic/skins/[skin]/load_skin.js&gt;.
// If you're using the <code>loadISC</code> tag, you can specify a locale-specific skin like so:
// <pre>
// &lt;isomorphic:loadISC skin="[localeName]"/&gt;
// </pre>
// Or you can use a script tag like so:
// <pre>
// &lt;SCRIPT SRC=/isomorphic/skins/[localeName]/load_skin.js&gt;&lt;/SCRIPT&gt;
// </pre>
// Where the <code>[localeName]</code> is the name of the current locale - you'll have this in the
// current page context as you'll need it to use the JSTL &lt;fmt:setLocale&gt; tag.
// <p>
// </var>
// Resources:
// <p>
// <b><u>Java</u></b>
// <ul>
// <li>Sun's i18n tutorial: +externalLink{http://java.sun.com/docs/books/tutorial/i18n/}
// <li>JSTL home page: +externalLink{http://java.sun.com/products/jsp/jstl/}
// <li>Apache JSTL "Getting Started": 
//     +externalLink{http://jakarta.apache.org/taglibs/doc/standard-doc/standard/GettingStarted.html}
// <li>Apache taglibs: +externalLink{http://jakarta.apache.org/taglibs/}
// </ul>
// <p>
//
// <b><u>.NET</u></b>
// <ul>
// <li>MSDN Developer Resource: "Developing World-Ready Applications":
// +externalLink{http://msdn.microsoft.com/library/default.asp?url=/library/en-us/cpguide/html/cpcondesigningglobalapplications.asp}
// </ul>
//
// <b><u>PHP</u></b>
// <ul>
// <li>PHP Gettext manual: +externalLink{http://us2.php.net/gettext}
// <li>O'Reilly Gettext tutorial: +externalLink{http://www.onlamp.com/pub/a/php/2002/06/13/php.html}
// </ul>
// <p>
//
// <b><u>Perl</u></b>
// <ul>
// <li>Perl, Unicode and i18n FAQ: +externalLink{http://rf.net/~james/perli18n.html}
// </ul>
// @treeLocation Concepts
// @title Internationalization and Localization
// @visibility external
//<





//> @groupDef dataSourceLocalization
// <var class="smartclient">
// DataSources can be created in +link{group:dataSourceDeclaration,several ways}.
// DataSources created directly in JavaScript can be 
// internationalized via the techniques described in the main +link{group:i18n,i18n article}.
// </var>
// DataSources which are declared in XML (.ds.xml files) and are read by the SmartClient 
// server, which are normally loaded <var class="smartclient">into a .jsp page via the 
// <code>&lt;isomorphic:loadDS&gt;</code> JSP tag,</var>
// <var class="smartgwt">by the <code>DataSourceLoader</code> servlet,</var> can instead be 
// internationalized using an approach similar to the internationalization of JSP files with
// JSTL tags.  This approach is also supported for screens defined using 
// +link{group:compponentXML,Component XML}.
// <p>
// <b>Note: </b>The tags we use for internationalizing SmartClient XML files look like 
// standard JSTL tags; this is intentional, simply because developers are familiar with JSTL.  
// However, the tags are being processed by SmartClient code, <b>not</b> JSTL, so only the 
// specific tags documented here are supported.
// <P>
// Given the following DataSource located in /shared/ds/supplyItem.ds.xml:
// <pre>
// &lt;DataSource ID="supplyItem"&gt;
//     &lt;fields&gt;
//         &lt;field name="itemName"&gt;
//             &lt;title&gt;Item Name&lt;/title&gt;
//             &lt;validators&gt;
//                 &lt;Validator type="lengthRange" max="40"&gt;
//                     &lt;errorMessage&gt;Must be 40 characters or less.&lt;/errorMessage&gt;
//                 &lt;/Validator&gt;
//             &lt;/validators&gt;
//         &lt;/field&gt;
//     &lt;/fields&gt;
// &lt;/DataSource&gt;
// </pre>
// To localize the title and validator error string of the <code>itemName</code> field, 
// change the DataSource definition as follows:
// <pre>
// &lt;DataSource ID="supplyItem" xmlns:fmt="WEB-INF/"&gt;
//     &lt;fields&gt;
//         &lt;field name="itemName"&gt;
//             &lt;title&gt;&lt;fmt:message key="itemTitle"/&gt;&lt;/title&gt;
//             &lt;validators&gt;
//                 &lt;Validator type="lengthRange" max="40"&gt;
//                     &lt;errorMessage&gt;&lt;fmt:message key="itemLengthRangeValidator"/&gt;&lt;/errorMessage&gt;
//                 &lt;/Validator&gt;
//             &lt;/validators&gt;
//         &lt;/field&gt;
//     &lt;/fields&gt;
// &lt;/DataSource&gt;
// </pre>
// This will cause SmartClient Server to look for a 
// <a href=http://docs.oracle.com/javase/1.4.2/docs/api/java/util/ResourceBundle.html#getBundle%28java.lang.String,%20java.util.Locale%29>ResourceBundle</a> 
// called "supplyItem", containing keys "itemTitle" and "itemLengthRangeValidator", and 
// replace the <code>&lt;fmt:message&gt;</code> tags with the values from the resource bundle
// in the expected way.  It obtains the user's <code>Locale</code> from the servlet request, 
// but you can override this if you want to force an application-specific locale, regardless of 
// the user's operating system settings.  To do this, <var class="smartclient"> either specify
// a "locale" attribute in the &lt;loadDS&gt; tag, like this:<br>
// <code>&lt;loadDS ID="myDataSource" locale="es" /&gt;</code><p>
// or </var> specify a "locale" parameter on HTTP requests to the <code>DataSourceLoader</code>
// and <code>IDACall</code> servlets.
// <p>
// <var class="smartclient"> In both of these cases, the </var>
// <var class="smartgwt">The </var> locale parameter should be an underscore-separated string
// conforming to the rules described in 
// <a href=http://docs.oracle.com/javase/1.4.2/docs/guide/intl/locale.doc.html>this article</a>
// on Java internationalization.  For example, "fr" (French language) or "en_US" (English 
// language, US location).
// <p>
// As mentioned, SmartClient Server will look for a <code>ResourceBundle</code> called 
// "supplyItem" in this case because it defaults to the name of the DataSource.  It is 
// possible to override this default at both the DataSource and field levels:
// <ul>
// <li>Specify <code>&lt;fmt:bundle&gt;</code> as a top-level DataSource tag, like this:<br>
// <pre>   &lt;DataSource  xmlns:fmt="WEB-INF/" ID="supplyItem"&gt;
//      &lt;fmt:bundle basename="com.isomorphic.test.i18n" /&gt;
//      ...
// &lt;/DataSource&gt;</pre></li>
// <li>Specify the bundle name in the individual <code>&lt;fmt:message&gt;</code> tags, like this:<br>
// <pre>   &lt;title&gt;&lt;fmt:message key='title1' bundle="com.mycompany.MyProperties" /&gt;&lt;/title&gt;</pre></li>
// </ul>
// When you name a resource bundle manually like this, if you qualify the name it influences
// where we expect to find that resource bundle.  In the above example, we would look in the 
// <code>com.mycompany</code> package.  For unqualified names (including the default of the 
// DataSource name that we use in the absence of an override), we look in the so-called 
// "default package", which corresponds to the root of your classes directory or the root of 
// a .JAR file.
// <p>
// Note that the <code>xmlns:fmt</code> attribute in the DataSource definition is required by 
// the XML parser if you intend to use our <code>fmt:message</code> features.  However, the 
// actual value you use is unimportant as long as it is present.
// <p>
// Although these examples don't show it, note that it is also possible to internationalize
// DataSource-level values in the same way as field-level values - for example:<pre>
//   &lt;DataSource  xmlns:fmt="WEB-INF/" ID="i18nTest"&gt; 
//     &lt;title&gt;&lt;fmt:message key="dsTitle" /&gt;&lt;/title&gt;
//      ...
// &lt;/DataSource&gt;
// </pre>
// <h4>Component XML</h4>
// All of the above applies similarly for localizing screens defined using 
// +link{group:componentXML,Component XML}.  For example:<pre>
// &lt;isomorphicXML xmlns:fmt="WEB-INF/"&gt;
//   &lt;fmt:bundle basename="com.isomorphic.test.i18n.test" /&gt;
//   &lt;Window ID="testWin1"&gt;
//     &lt;title&gt;&lt;fmt:message key='title1' /&gt;&lt;/title&gt;
//   &lt;/Window&gt;
// &lt;/isomorphicXML&gt;
// </pre>
// Note the following differences between localizing <code>.ds.xml</code> (DataSource) files and
// localizing <code>.ui.xml</code> (component XML) files: <ul>
// <li>The <code>&lt;isomorphicXML&gt;</code> tag, which is ordinarily only used to wrap 
// multiple widget definitions to give a valid XML file, is required if you are using this 
// internationalization technique.</li>
// <li>Because there is no concept of an "ID" associated with a <code>.ui.xml</code> file, 
// there is no default bundle like there is with DataSource definitions.  Instead, you have to
// specify the bundle by hand, either by adding a <code>&lt;fmt:bundle&gt;</code> tag as an 
// immediate child of the <code>&lt;isomorphicXML&gt;</code> tag, or by specifying 
// <code>bundle</code> attributes in your individual <code>&lt;fmt:message&gt;</code> tags.</li>
// <li>Locale can be overridden, as described for DataSources above, by 
// specifying a "locale" parameter on HTTP requests to the <code>ScreenLoaderServlet</code> 
// (this is done for you when you pass a locale to the +link{RPCManager.loadScreen} method).</li>
// </ul>
// @treeLocation Concepts/Internationalization and Localization
// @title DataSource and Component XML Localization
// @visibility external
//<

//> @groupDef localizedNumberFormatting
//
// Use the special +link{dataSourceField.type,field types} "localeInt", "localeFloat" and
// "localeCurrency" to have locale-specific formatting applied to numeric values.
// <p>
// <var class="smartclient">
// These special types rely on the settings +link{numberUtil.decimalSymbol},
// +link{numberUtil.groupingSymbol} and +link{numberUtil.currencySymbol}.  These are normally
// set automatically when you load a language pack for the current locale.
// <p>
// See +link{group:i18n} for more background on internationalization and loading language packs
// for the current locale.
// </var>
// <var class="smartgwt">
// These special types automatically use GWT's NumberFormat logic, using the default locale.
// Additional +link{SimpleType,SimpleTypes} can be created that call GWT's NumberFormat if more
// variations are needed.
// <p>
// Note that language packs shipped with SmartGWT contain properties such as "decimalSymbol"
// that seemingly would control localized number formatting.  These are inactive for SmartGWT
// and should be ignored.
// <p>
// See +link{group:i18n} for more background on internationalization.
// </var>
//
// @title Localized Number Formatting
// @visibility external
//<

//> @groupDef featureExplorerOverview
//
// The +docTreeLink{FeatureExplorer, Feature Explorer} is an example shell designed to help you
// explore the capabilities of SmartClient.  Read on for a brief overview, including specific
// instructions for using the example code in your own standalone application.
// <p>
// The tree on the left of the Feature Explorer contains examples grouped by logical
// categories.  Selecting an example brings it up on the right side of the screen, inside a
// tabbed view.  The default "View" tab shows the interactive example itself, while other
// tabs show the underlying source.  The "JS" tab shows the source for the example.  There is
// sometimes also an "XML" tab that shows the corresponding version in 
// +link{componentXML, Component XML} format.  For databound examples, there are also
// frequently additional tabs that show the +link{dataSource, DataSources} associated with the
// example.
// <p>
// <b><u>How to create a standalone application using code from the Feature Explorer</u></b>
// <p>
// The Feature Explorer shell is designed to show many examples in one place and to enable
// experimentation by providing a way for you to modify example code and try out your changes.
// As a result, the Feature Explorer requires some basic server support and its examples omit
// the usual SmartClient module includes that have to be in place for using SmartClient
// components standalone.
// <p>
// If you'd like to use example code in your application to get started quickly, create a page
// with SmartClient includes and then take the code from the "JS" tab and place it between
// &lt;SCRIPT&gt; blocks as described +link{group:nonJavaBackend, here}.  If the example also
// includes a datasource, place the datasource definitions in the same file before the
// component code.  Note that DataSources (and components) written in XML require the optional
// SmartClient server.  If you're using the server, you can include them on your page using the 
// +link{group:loadDSTag} tag.
// <p>
// <b><u>Feature Explorer difference in the LGPL package</u></b>
// <p>
// The LGPL edition of SmartClient does not include the SmartClient Java Server as part of the
// licensed software, but a trimmed down server is included in the package to support the
// Feature Explorer shell.  There are some examples that use DataSources that would normally
// use the SmartClient server for persistence.  In the LGPL package, these DataSources are
// automatically turned into +link{group:clientOnlyDataSources, Client Only DataSources} and
// the Feature Explorer loads the data for these one-time from the dataURL or testFileName
// attributes specified on the DataSource.  Subsequent DataSource operations work against this
// client-side dataset, which is why changes to the data aren't permanent in these examples.
// <p>
// Check out the +link{clientServerIntegration, Client-Server Integration} overview topic for
// an overview of your DataBinding options.
//
// @title Feature Explorer Overview
// @visibility external
//<

//> @groupDef iscInstall
//
// This overview serves as a how-to for installing SmartClient into your web application.
// Evaluators are urged to use the SmartClient SDK with the embedded tomcat servlet engine
// during evaluation rather than pursue installation into an existing web application up front,
// however, reading this document and the related +link{group:clientServerIntegration,server
// integration} materials is recommended to get an overview.
// <P>
// SmartClient has two pieces - the client components that run in the browser and the server
// components that run in a J2SE-compatible container.  You don't need to use a Java back-end
// to use SmartClient, but the SDK comes with some examples that assume the presence of the
// Java back-end and, for some examples, a SQL Database.  If you will be using SmartClient with
// a Java back-end, see below for the list of J2SE application servers supported by the Java
// implementation of the SmartClient server.
// <p>
// The SDK contains two top-level directories: <code>smartclientSDK</code> and
// <code>smartclientRuntime</code>.  The <code>smartclientSDK</code> directory contains the
// embedded servlet engine, embedded database, examples, and documentation.  The
// <code>smartclientRuntime</code> directory contains just the client and server components of
// the SmartClient product - use the contents of this directory when deploying SmartClient into
// your application environment.
// <p>
// <u><b>Client integration</b></u>
// <p>
// To install the client-side portion of SmartClient, simply copy the <code>isomorphic</code>
// directory from the smartclientRuntime webroot to the webroot of your application.  Having
// done this you can use SmartClient components on your pages regardless of the technologies
// used on your back-end and you can bind to server-side componentry backed by arbitrary
// technology - see the <i>Data Integration</i> section of the
// +link{group:clientServerIntegration} section for more information.
// <p>
// <u><b>Server integration</b></u>
// <p>
// <span style='color:red'>Note:</span> Some of the instructions below ask you to copy files
// into the WEB-INF/classes folder.  If you're using an IDE such as Eclipse that attempts to
// manage the WEB-INF/classes folder, we recommend that you copy these files to the src/
// directory of your project (next to the top-level folder for your java namespace) such that
// your IDE auto-deploys them to the WEB-INF/classes folder.  We have seen cases of tools like
// Eclipse periodically deleting files that are checked into to WEB-INF/classes directly.
// <p>
// <ul>
// <li> Copy all files from the WEB-INF/lib directory of the smartclientRuntime to your
// WEB-INF/lib.  The set of libs in the smartclientRuntime/WEB-INF/lib folder is a minimal set;
// smartclient<b>SDK</b>/WEB-INF/lib contains all other .jars you might need, including third-party
// libraries bundled for convenience.  See
// +link{group:javaModuleDependencies,Java Module Dependencies} for details of the .JAR files that
// comprise the SmartClient Server, and their dependencies on various third-party libraries.
// Generally, if there are conflicts with the versions of third-party libraries you want to
// use, you can use the versions you want - SmartClient has minimal dependencies on these
// libraries. 
// <li> Copy the WEB-INF/classes/log4j.isc.config.xml from the smartclientRuntime to your
// WEB-INF/classes
// directory.  This file contains the SmartClient server log configuration.
// <li> Copy the +link{server_properties,WEB-INF/classes/server.properties}
// from the smartclientRuntime to your WEB-INF/classes
// directory.  This file contains settings for basic file locations such the location of
// webroot, the SmartClient SQL engine and DMI.  The version under smartclientRuntime has a
// basic, secure configuration.  See the version of +link{server_properties,server.properties}
// under the smartclientSDK directory for sample SQL and other settings.
// <li> Copy the WEB-INF/iscTaglib.xml from the smartclientRuntime to your WEB-INF directory.
// This file enables the use of custom SmartClient tags in your JSPs.  
// <li> Merge portions of the WEB-INF/web.xml into your application's web.xml.  To use
// SmartClient JSP tags like &lt;loadISC&gt;, you'll need to merge the &lt;jsp-config&gt;
// section from web.xml.  Also there are some mandatory and optional servlets and filters to
// merge - see below.
// <li> <b>Power and Enterprise Editions only</b>.  Copy the shared/ds/batchUpload.ds.xml file
// to the same location in your target webapp directory.  This file is a utility DataSource
// that is used to provide the initial upload functionality of the +link{BatchUploader} 
// component - strictly speaking, you only need to perform this step if you intend to use that 
// component.
// </ul>
// <p>
// See +link{servletDetails,Core and Optional SmartClient servlets} for details of additional 
// changes you may need to make to your applications <code>web.xml</code> file.  See 
// +link{group:javaModuleDependencies,Java Module Dependencies} for details of the .JAR files that
// comprise the SmartClient Server, and their dependencies on various third-party libraries.
// <p>
// <u><b>Multiple Applications / WARs</b></u>
// <p>
// To integrate the server portion of SmartClient, you need to follow the steps below for each
// application (WAR) that uses SmartClient.  Note that, if installing into an environment that
// uses multiple WARs, installation of SmartClient JARs into a directory shared by multiple
// applications is not supported.  Installation of a separate WAR with client-side SmartClient
// modules for maintaining cache coherence across applications using the same version of ISC is
// supported - contact Isomorphic support for more details on how to set that up.
// <p>
// <u><b>Troubleshooting</b></u>
// <p>
// This section covers some common problems with possible solutions.  You may also need to refer to the
// documentation for your specific application server, web server, or database. If you experience any problems
// installing and configuring SmartClient in your environment, please post on the 
// <a href="http://forums.smartclient.com/" target='_blank'>SmartClient forums</a> for
// assistance.
// <p>
// <table width="90%" class="normal" align="center" border="1" cellpadding="5">
// <tr bgcolor="#808080">
// 	<td width="30%"><b>Problem</b></td>
// 	<td width="30%"><b>Possible Causes</b></td>
// 	<td width="40%"><b>Solution</b></td>
// </tr><tr>
// 	<td rowspan='3'>Browser displays a generic "page cannot be displayed" or "unable to locate the server" message.</td>
// 	<td>Servlet engine not started.</td>
// 	<td>Start your application server.</td>
// </tr><tr>
// 	<td>Missing/incorrect port for servlet engine in URL.</td>
// 	<td>Check the startup messages, logs, or documentation for the servlet engine to determine what port it is using.</td>
// </tr><tr>
// 	<td>Host name is incorrect.</td>
// 	<td>Check whether other pages on the host can be accessed.
// 		Try the base URL http://[host name]:[port number] to see whether the servlet engine or webserver is functioning.</td>
// </tr><tr>
// 	<td>Browser displays a 404 or other page/file not found error.</td>
// 	<td>Incorrect URL.</td>
// 	<td>Check for errors in the URL, including capitalization.</td>
// </tr><tr>
// 	<td>Server error: taglib not defined</td>
// 	<td>Missing iscTaglib.xml or iscTaglib.xml not referenced in web.xml</td>
// 	<td>Copy WEB-INF/iscTaglib.xml from smartclientRuntime to your deployment WEB-INF directory
// 	and make sure that you have merged the &lt;jsp-config&gt; section from the
// 	smartclientRuntime web.xml</td>
// </tr><tr>
//  <td>ClassNotFound or other Java Exceptions in the server log.</td>
//  <td>Missing JAR files</td>
//  <td>Verify every required .jar has been copied into the WEB-INF/lib directory of your
//  deployment.  Use +link{group:javaModuleDependencies,these docs} to double-check.  If in
//  doubt, copy every available .jar, verify this is working, then trim off .jars you are
//  definitely not using.</td>
// </tr><tr>
//  <td>"isc" is not defined JS error</td>
//  <td>Incorrect URLs to SmartClient modules</td>
//  <td>Use View Source to look at SCRIPT includes (e.g. for ISC_Core.js), try those URLs
//  directly in the browser to verify the files are correctly deployed</td>
// </tr>
// </table>
// <P>
// <u><b>Caching Considerations</b></u>
// <p>
// When upgrading from one SmartClient release to the next, you want to make sure that the user
// picks up the new version on next access, but you also want to keep the ISC modules cacheable
// so they're not refetched on every access.
// <p>
// SmartClient deals with this problem by appending a version string as a query parameter to
// each module load directive.  This is done by the &lt;isomorphic:loadISC&gt; and
// &lt;isomorphic:loadModules&gt; tags automatically.  As long as you make sure that the file
// that contains these tags is non-cacheable, you will get the desired behavior.
// <p>
// <u><b>Supported J2SE Containers</b></u>
// <p>
// Below is the list of J2SE containers that have been tested to be compatible with this
// version of SmartClient.  Installation in these containers is supported for deployment by
// Isomorphic.  If your application server is not on this list, please contact us at the
// <a href="http://forums.smartclient.com" target='_blank'>SmartClient forums</a> to see if we can support
// your deployment.  In general, the Java portion of ISC should work on servlet containers that
// comply with servlet specification version 2.3 and up and utilize a JVM no older than version
// 1.4.
// <p>
// Supported J2SE Containers:
// <p>
// <table class='normal'>
// <tr><td width=40></td><td width=400><i>Apache Tomcat 4.0.x, 4.1.x, 5.0.x, 5.5x, 6.0.x, 7.0.x, 8.0x</i></td><td></td></tr>
// <tr><td></td><td><i>Apache Geronimo 1.x, 2.x, 3.x</i></td><td></td></tr>
// <tr><td></td><td><i>Oracle WebLogic 6.x, 7x, 8.x, 9.x, 10.x, 11gR1 PSx, 12c Release x</i></td><td></td></tr>
// <tr><td></td><td><i>Caucho Resin 2.1.x, 3.0.x, 3.1.x, 4.x</i></td><td></td></tr>
// <tr><td></td><td><i>IBM WebSphere 5.x, 6.x, 7.x, 8.x</i></td><td></td></tr>
// <tr><td></td><td><i>IBM WebSphere Community Edition 1.x, 2.x, 3.x</i></td><td></td></tr>
// <tr><td></td><td><i>JBoss 3.2.x, 4.0.x, 4.2.x, 5.x, 7.x, 8.x</i></td><td></td></tr>
// <tr><td></td><td><i>Mortbay Jetty 4.x, 5.x, 6.x, 7.x, 8.x, 9.0.x</i></td><td></td></tr>
// <tr><td></td><td><i>Oracle Containers for J2EE (OC4J) 9.x, 10.x, 11.x</i></td><td></td></tr>
// <tr><td></td><td><i>Oracle Application Server 10g 9.x, 10.x; 11g</i></td><td></td></tr>
// <tr><td></td><td><i>Sun Application Server 8.x, 9.x</i></td><td></td></tr>
// </table>
//
// @treeLocation Concepts
// @title Deploying SmartClient
// @visibility external
//<


//> @groupDef sgwtEESetup
//
// <h3>Creating a new project from scratch</h3>
// <P>
// If you are starting a new project from scratch, we recommend starting with one of the sample
// projects included with SmartGWT EE under the "samples" directory.  Use the project that most
// closely matches your integration strategy and follow the instructions in the "readme" file
// within the sample directory.
// <P>
// <h3>Adding SmartGWT EE to an existing project</h3>
// <P>
// <B>Important:</b> If you have created a project using the GWT Plugin for Eclipse or a
// similar tool, first get rid of unused resources typically included in such "starter projects":
// <ul>
//   <li> from your [i]moduleName[/i].gwt.xml file, remove imports of any GWT themes, leaving
//        only the import of com.google.gwt.user.User.  Proper imports to add for SmartGWT
//        Pro/EE are shown below.
//   <li> get rid of any sample servlets or GWT-RPC services (delete both web.xml entries and
//        server-side source)
// </ul>
// <P>
// Now add SmartGWT EE: all of these steps are required.  <b>Do not skip steps or omit
// resources you think you may not need</b>.  Get a working project first, then think about
// trimming resources, as needed, once you understand the system.
// <P>
// In the steps below, copying from eg "war/WEB-INF/..." means copying from the "war"
// subdirectory of whichever sample project is closest to your desired integration strategy.
// <P>
// <ol>
// <li>add all jars from the SmartGWT Pro/EE distribution (under smartgwtee-<i>version</i>/lib)
//     to your CLASSPATH.  You can copy the '.classpath' file from the appropriate sample
//     project for a working set of includes (it's in the same directory as the <i>war/</i>
//     folder). There are both client- and server-side jars, both kinds go in the classpath.
//     For server-side .jars, dependencies are documented +link{group:javaModuleDependencies,here}.
// <li>update your <i>modulename</i>.gwt.xml file with these imports:
//    <pre>
//    &lt;inherits name="com.smartgwtee.SmartGwtEE"/&gt;
//    &lt;inherits name="com.smartgwtee.tools.Tools"/&gt;
//    </pre>
//    <b>NOTE:</b> if you were previously using SmartGWT LGPL, <b>remove</b> the
//    &lt;inherit&gt; of 'com.smartgwt.SmartGWT' and ensure the &lt;inherit&gt; of
//    'com.smartgwt.tools.SmartGwtTools' appears <b>before</b> the imports above.  Also ensure
//    that all SmartGWT-related inherits appear *before* your &lt;entry-point&gt; declaration.
// <li> if you are using Optional Modules such as Analytics or RealTimeMessaging, see
//      +link{group:loadingOptionalModules} for additional inherits
// <li> Copy log4j.isc.config.xml across to the "src" dir of your project (it's
//      placed in the "src" dir as a means of getting it into the CLASSPATH).  This file is
//      either in the "src/" dir of a given sample project or in war/WEB-INF/classes.  This
//      enables default log4 categories for server-side logs appropriate for development.
// <li> Copy +link{server_properties,server.properties} across to the "src" dir of your project.
//      This file is either in the "src/" dir of a given sample project or in war/WEB-INF/classes.
//      This contains miscellaneous server settings - see the file itself for documentation.
//    <ul>
//       <li><b>server.properties contains server-side paths that contain the GWT module name</b>.
//           Replace the module name from the sample project (eg "builtinds") with the module
//           name of your actual GWT project 
//       <li> if you're using SQLDataSource, enter your JDBC settings (start with a
//            server.properties from a project that uses a database to see the required format for
//            settings)
//    </ul>
// <li> If you plan to use JPA, copy src/META-INF/persistence.xml into your src/META-INF directory.
//      This file will be copied into war/WEB-INF/classes/META-INF directory during build process.
//      Make sure you specify your entity classes in persistence.xml. If you change persistence unit
//      name you have to update "jpa.persistenceUnitName" property in 
//      +link{server_properties,server.properties} file. More
//      information on JPA configuration can be found in +link{group:jpaIntegration}.
// <li> merge web.xml, starting with the web.xml from the sample project that is closest to
//      your target application (under war/WEB-INF in the sample project).  The list of
//      servlets and what they do is documented +link{group:servletDetails,here}.  See the
//      server-side JavaDoc for further details on servlet APIs and override points.
// <li> Copy war/WEB-INF/iscTaglib.xml to your war/WEB-INF/iscTaglib.xml.  This is required for
//      JSP tags to work and used by tools such as Visual Builder.  Be sure you have also
//      copied the web.xml section that enables the taglib (&lt;jsp-config&gt; element) 
// <li> (SmartGWT Power or Enterprise only) If using the BatchUpload functionality, copy
//      war/ds/batchUpload.ds.xml from the Showcase sample project to the shared.datasources
//      directory you configured in +link{server_properties,server.properties}.
// <p>
// <u><b>Troubleshooting</b></u>
// <table width="90%" class="normal" align="center" border="1" cellpadding="5">
// <tr bgcolor="#808080">
// 	<td width="30%"><b>Problem</b></td>
// 	<td width="30%"><b>Possible Causes</b></td>
// 	<td width="40%"><b>Solution</b></td>
// </tr><tr>
//  <td>ClassNotFound or other Java Exceptions in the server log.</td>
//  <td>Missing JAR files</td>
//  <td>Verify every .jar from the smartgwtee-<i>version</i>/lib directory has been added to your CLASSPATH.
//  Although you might later be able to remove some .jars, for initial installation testing,
//  copy every .jar</td>
// </tr><tr>
// 	<td>Client-side error about attempting to use "iscServer" request without server installed</td>
// 	<td>Wrong imports in <i>moduleName</i>.gwt.xml</td>
// 	<td>See correct imports above, note <b>remove</b> com.smartgwt.SmartGWT import and be sure
//      imports are in the correct order</td>
// </tr><tr>
// 	<td>Client-side error about "$debox is not defined"</td>
// 	<td>Wrong order of imports in <i>moduleName</i>.gwt.xml</td>
// 	<td>Make sure all SmartGWT-related imports appear before &lt;entry-point&gt; in your
//      .gwt.xml file</td>
// </tr><tr>
// 	<td>Missing images or failure to load page</td>
// 	<td>Didn't set isomorphicDir in .html bootstrap file</td>
// 	<td>See step above for setting isomorphicDir</td>
// </tr><tr>
// 	<td>Server error: "adminConsole" app not found when launching tools such as Visual Builder</td>
// 	<td>Bad filesystem paths configured in +link{server_properties,server.properties}</td>
// 	<td>Correct paths in +link{server_properties,server.properties}.  
//      <b>NOTE</b> the samples include the GWT module
//      name in some settings, search and replace this value with your GWT module name</td>
// </tr><tr>
// 	<td>Server error: Can't find DataSource with ID <i>yourDataSourceID</i></td>
// 	<td>Bad filesystem paths in +link{server_properties,server.properties} or 
//      bad DataSource .ds.xml file</td>
// 	<td>Correct paths in +link{server_properties,server.properties} - search for 
//      "project.datasources".  Also check that
//      the DataSource ID you are using matches the "ID" attribute in the .ds.xml file (NOTE:
//      ID attribute is uppercase "ID" not "id") and that the file is named
//      <i>dataSourceId</i>.ds.xml.  DataSource IDs are <b>case sensitive</b>, including the
//      file name</td>.  For more in-depth troubleshooting steps, see 
//      +link{http://forums.smartclient.com/showthread.php?t=8159#aDSLoad,this FAQ answer}.
// </tr><tr>
// 	<td>Server error: taglib not defined</td>
// 	<td>Missing iscTaglib.xml or iscTaglib.xml not referenced in web.xml</td>
// 	<td>See step above about iscTaglib.xml and web.xml entry</td>
// </tr>
// </table>
// <P>
// For further troubleshooting steps, see
// +externalLink{http://forums.smartclient.com/showthread.php?t=8159,the SmartGWT FAQ} and, if
// still stuck, try posting in the Forums. <B>NOTE:</B> gather all the information indicated in
// the FAQ before posting.
// <p>
// <b>NOTE:</b>  It is possible to create a server-only SmartGWT EE project that will run 
// standalone (ie, not inside a servlet container or application server).  Please see
// +link{groupDef:standaloneDataSourceUsage,Standalone DataSource Usage} for details.
// 
// @title Smart GWT Pro/EE Project Setup & Deployment
// @visibility sgwt
//<

//> @groupDef server_properties
//
// The <code>server.properties</code> file is a configuration file read by the SmartClient
// server. It contains miscellaneous server settings - see the file itself for documentation 
// and default or sample settings.
// <P>
// Note that this is a standard Java Properties file, except it allows variable substitution
// from other properties defined earlier in the file.
// <P>
// It's loaded from the <code>CLASSPATH</code>, so it can be anywhere in the 
// <code>CLASSPATH</code>, but is typically either in the projects java "src" directory or
// in <code>WEB-INF/classes</code>.
// <P>
// Server side code can access and modify the properties specified in this file via the
// <code>com.isomorphic.base.Config</code> class
//
// @visibility external
// @title <code>server.properties</code> file
// @treeLocation Concepts/Deploying SmartClient
//<

//> @groupDef servletDetails
// 
// The following is a description of the servlets and filters you'll find in the web.xml file
// contained in the smartclientRuntime and what they do:
// <p>
// <i>Core Functionality</i>
// <ul>
// <li>Init servlet - see the article on +link{group:serverInit,SmartClient Server Initialization</li>
// <li>IDACall servlet - <b>required</b> for +link{dmiOverview,DMI}, built-in RPC operations and built-in DataSource
// operations to work.  All databound examples in the SDK use this servlet.  If you're planning on
// using a custom actionURL for all your RPC requests, then you don't need this servlet.
// <li>FileDownload servlet - required for serving the Isomorphic framework code compressed and with
// caching headers as well as for serving skin images with caching headers.  It is highly
// recommended that you use this for production but is not required. 
// <li>PreCache servlet - loads resources into memory on container startup.  Not required, but if you
// exclude this servlet there may be a slow response to the very first request.
// <li>jsp-config section - the iscTaglib registration block is required to use
// &lt;isomorphic&gt; tags, and the *.isc and *.rpc mappings.  These are optional, if you want
// to use these as handy development tools.
// </ul>
// <p>
// <i>Optional Functionality</i>
// <ul>
// <li>RESTHandler servlet - handles SmartClient Server DataSource operations issued by 
// REST clients: it's like IDACall, but for the REST protocol.  Typically,  the clients
// of this servlet would not be ordinary SmartClient/SmartGWT applications (though they 
// could be), but other client technologies that need to access SmartClient DataSource 
// operations as reusable services.  If you do not plan to connect to the server using the
// REST protocol, then you don't need this servlet.
// <li>AxisServlet - exposes all DataSource operations via a WSDL service described by 
// SmartClientOperations.wsdl.  This is effectively the same as the RESTHandler servlet, but 
// for SOAP clients.  If you do not plan to connect to the server using webservice protocols,
// then you don't need this servlet.
// <li>HttpProxy - used by the RPCManager when sending AJAX RPCs to a server other than the
// server that serves the main application page.  You need to install this servlet if, for
// example, your application will be querying web services exposed by servers other than the
// server that is serving the rest of the application.  See the javadoc for this servlet for
// various configuration options, such as how to restrict the URLs that are allowed to be
// proxied. 
// <li>MessagingServlet - used by the realtime messaging system.  If you're planning
// on using this subsystem, you'll need this servlet.
// <li>CompressionFilter - required if you want to use dynamic compression of html and js
// files.
// <li>JSSyntaxScannerFilter - development tool that looks for trailing commas in JS
// source (scans html files for &lt;script&gt; tags and scans .js files in their entirety).
// This is a useful development tool, but should not be included in production.
// <li>NoCacheFilter - development tool that makes any content it intercepts non-cacheable in
// order to ensure developers are looking at the latest version of a file when modifying
// examples.  Not for production use.
// <li>DataSourceLoader - a servlet that returns the definition of one or more DataSources in
// JavaScript notation.  This servlet is provided as an alternative to using the 
// <code>&lt;isomorphic:loadDS&gt;</code> JSP tag, and is particularly suitable in environments
// where JSP tags can't be used for some reason (such as with SmartGWT).  See 
// +link{group:dataSourceDeclaration,Creating DataSources} for more details.
// <li>ScreenLoaderServlet - a servlet that returns the definition of one or more screens in
// JavaScript notation.</li>
// </ul>
// Note that not all of the servlets and filters listed under <i>Optional Functionality</i>
// above are present in the web.xml that ships with the smartclientRuntime - if you need to use
// any of these, copy their configuration from the web.xml available under the WEB-INF
// directory of smartclientSDK.  Other servlets, filters and configuration files from the
// smartclientSDK should not be copied to your deployment, simply because the SDK includes many
// developer tools that are not extensively audited from a security standpoint.
//
// @treeLocation Concepts/Deploying SmartClient
// @title The Core and Optional SmartClient servlets
// @visibility external
//<


//> @groupDef serverInit
// 
// The SmartClient Server framework must be initialized from its config files at startup
// time, before user code invokes <em>any</em> framework functionality.  If you are running
// standalone (ie, outside of a servlet container), you should call the static utility method 
// <code>ISCInit.go()</code> early in your bootstrap code (eg, from the top of your 
// <code>main()</code> method).
// <p>
// If you are running inside a servlet engine, there are two ways to initialize the framework:
// <ul>
// <li>Install <code>InitListener</code>, which is a <code>ServletContextListener</code>:<br>
// <pre>   &lt;listener&gt;
//        &lt;listener-class&gt;com.isomorphic.base.InitListener&lt;/listener-class&gt;
//    &lt;/listener&gt;</pre></li>
// <li>Install <code>Init</code>, which is a <code>Servlet</code>, to load at startup:<br>
// <pre>    &lt;servlet&gt;
//        &lt;servlet-name&gt;Init&lt;/servlet-name&gt;
//        &lt;servlet-class&gt;com.isomorphic.base.Init&lt;/servlet-class&gt;
//        &lt;load-on-startup&gt;1&lt;/load-on-startup&gt;
//    &lt;/servlet&gt;</pre></li>
// </ul>
// As shipped, the framework's <code>web.xml</code> file loads both of these classes, to ensure 
// the best chance of correct and early initialization.  The recommended approach is to use 
// <code>InitListener</code> if your servlet container implements the Servlet 2.4 API or 
// greater, because <code>ServletContextListener</code>s are guaranteed to run before any 
// servlets or filters are instantiated.  <code>ServletContextListener</code>s are available
// in Servlet API 2.3, but the requirement that they run before any filter or servlet is 
// initialized was not added until 2.4.  Some 2.3 servlet engines do enforce this behavior 
// even though it is not part of the spec; experimentation with your servlet container of 
// choice will be required if it only implements 2.3.
// <p>
// Note that it does no harm to leave the <code>web.xml</code> file with both <code>Init</code>
// and <code>InitListener</code> in place, because the framework simply ignores any request 
// to initialize once initialization has taken place.
// <p>
// <h4>Interaction with Spring initialization</h4>
// The Spring framework attempts to initialize itself in similar ways to the SmartClient
// framework, and similarly it wants its own initialization to be the very first thing that 
// happens.  This can cause problems in certain cases.  For example, one known problem occurs
// when you define DataSource instances as Spring beans, and Spring gets to initialize itself 
// first: it attempts to instantiate a instance of the bean, which ultimately calls into 
// our framework code before the framework has been initialized.
// <p>
// The solution to this is to use the <code>InitListener</code>, and to ensure that it is 
// declared in your <code>web.xml</code> file before the Spring
// <code>ContextLoaderListener</code>.  The Servlet 2.3 spec states (SRV.10.3.3):<p>
// <em>&nbsp;&nbsp;"During Web application execution, listeners are invoked in the order of their 
// registration."</em>
// <p>
// Therefore, declaring the SmartClient listener before the Spring one ensures that
// things run in the correct order in Servlet 2.4-compatible containers, and in those Servlet
// 2.3 containers that enforce the rule of running listeners before initializing filters or
// servlets.
//
// @treeLocation Concepts/Deploying SmartClient
// @title Server Framework Initialization
// @visibility external
//<

//> @groupDef javaModuleDependencies
//
// The following is a description of what functionality is contained in each SmartClient JAR
// file, and which other SmartClient JARs and third party libraries are required to enable that
// functionality.  Note that actual filenames in WEB-INF/lib will typically contain the version
// number of the library, whereas the library names below simply list the base name.
// <P>
// All of the libraries that SmartClient depends upon have either an Apache 1.0 or
// Apache 2.0 license, with the following exceptions:<br>
// 1. Optional Hibernate support requires LGPL-licensed .jars from Hibernate<br>
// 2. Optional bean annotation support requires an implementation of javax.persistence.jar.
// This is available from several providers - for example, SpringSource, Hibernate or the 
// Glassfish project.  The licensing of this library varies by provider.<br>
// 3. Optional support for Java as a JSR223 server-side scripting language is provided by
// software developed by Isomorphic, but based on the "java-engine" package, originally 
// developed by Sun.  See +link{sunNotice}.
// <P>
// If you are using GWT, GWT itself also has an Apache 2.0 license, however tools and
// test environments that you may use during development have different license (such as
// Eclipse - Eclipse Public License).  Also, specific GWT widgets, not required by Smart GWT, 
// have licenses different from core GWT (such as JFreeChart's LGPL license).  See 
// +externalLink{http://code.google.com/webtoolkit/terms.html,Google's summary of terms} for
// details.
// <ul>
// <li><b>isomorphic_core_rpc</b>: This is the core SmartClient module.  It provides the RPC, DMI,
// and DataSource support.<br>
// &nbsp;&nbsp;<u>Requires</u>:<br>
// &nbsp;&nbsp;&nbsp;&nbsp;commons-cli<br>
// &nbsp;&nbsp;&nbsp;&nbsp;commons-lang<br>
// &nbsp;&nbsp;&nbsp;&nbsp;commons-collections<br>
// &nbsp;&nbsp;&nbsp;&nbsp;commons-pool<br>
// &nbsp;&nbsp;&nbsp;&nbsp;commons-codec<br>
// &nbsp;&nbsp;&nbsp;&nbsp;isc-jakarta-oro<br>
// &nbsp;&nbsp;&nbsp;&nbsp;log4j<br>
// &nbsp;&nbsp;&nbsp;&nbsp;commons-jxpath<br>
// &nbsp;&nbsp;&nbsp;&nbsp;commons-httpclient<br>
// &nbsp;&nbsp;&nbsp;&nbsp;commons-vfs<br>
// &nbsp;&nbsp;&nbsp;&nbsp;velocity<br>
// &nbsp;&nbsp;&nbsp;&nbsp;commons-fileupload<br>
// &nbsp;&nbsp;<u>Optionally Requires</u>:<br>
// &nbsp;&nbsp;&nbsp;&nbsp;isomorphic_js_parser - if you're using the built-in support for REST via the RESTHandler servlet with JSON payloads<br>
// &nbsp;&nbsp;&nbsp;&nbsp;xercesImpl - if you're using JDK &lt; 1.5<br>
// &nbsp;&nbsp;&nbsp;&nbsp;xml-apis - if you're using JDK &lt; 1.5<br>
// &nbsp;&nbsp;&nbsp;&nbsp;mail - if you plan to use the Mail messaging feature<br>
// &nbsp;&nbsp;&nbsp;&nbsp;activation - if you plan to use the Mail messaging feature and you
// are using a JDK &lt; 1.6<br><br>
// &nbsp;&nbsp;&nbsp;&nbsp;javax.persistence - if you plan to use the metadata-from-annotations 
// feature.  Note that if you are using JPA, or a recent version of Hibernate, then you are
// probably already using this library.<br><br>
// &nbsp;&nbsp;&nbsp;&nbsp;poi - if you plan to export datasets in Microsoft Excel 97 (xls)  
// or 2007 (xlsx) formats.  Additionally, if you plan to export data in Excel 2007 (xlsx) 
// format, you will need the following libraries:<br>
// &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;poi-ooxml, poi-ooxml-schemas, xbean, jsr173_1.0_api<br><br>
// &nbsp;&nbsp;&nbsp;&nbsp;PDF Export - Additional .jars are required if PDF Export is to be used. These .jars are:<br>
// &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;core-render, iText-2.0.8, jtidy-r938<br>
// &nbsp;&nbsp;&nbsp;&nbsp;Image Export and IE6-8 DrawPane PDF Export - These libraries are required to use
// +link{RPCManager.exportImage()}, or when using +link{RPCManager.exportContent()} to export a DrawPane or FacetChart 
// only in IE8 or earlier:<br>
// &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;batik-anim, batik-awt-util, batik-bridge, batik-css,
// batik-dom, batik-ext, batik-gvt, batik-parser, batik-script, batik-svg-dom, batik-util, batik-xml,
// xml-apis-ext<br><br>
//
// &nbsp;&nbsp;&nbsp;&nbsp;<b>NOTE:</b> Between iText 2.0.x and iText 2.1.x there is a binary
// (but not source) incompatibility that causes a server-side <code>NoSuchMethodError</code> when exporting,
// e.g., charts in Internet Explorer. This is a +externalLink{http://code.google.com/p/flying-saucer/issues/detail?id=126,known issue}
// with the Flying Saucer product that is fixed by using core-renderer-R8-isomorphic.jar and iText-2.1.7.jar in the
// lib-iTextAlternate/ directory instead of core-renderer.jar and iText-2.0.8.jar in the lib/ directory.
// To use iText 2.1.7 with the Server Framework, exclude lib/core-renderer.jar and lib/iText-2.0.8.jar
// from the application's classpath and instead add lib-iTextAlternate/core-renderer-R8-isomorphic.jar and
// lib-iTextAlternate/iText-2.1.7.jar.<br><br>
//
// &nbsp;&nbsp;&nbsp;&nbsp;isomorphic_jpa and its dependencies - if you plan to use BatchDS Generator even if you are
// not using JPA, although the generated DataSources will not require JPA at runtime if you are not using JPA.
// </li>
// <p>
// <li><b>isomorphic_contentexport.jar</b>: Optional support for PDF Export requires Flying Saucer, iText and jTidy, introducing:<br>
// &nbsp;&nbsp;- Flying Saucer: core-render.jar: +externalLink{http://www.gnu.org/licenses/lgpl-2.1.html,LGPL license}<br>
// &nbsp;&nbsp;- iText: iText-2.0.8.jar: +externalLink{http://www.mozilla.org/MPL/1.1/,Mozilla Public License Version 1.1}<br>
// &nbsp;&nbsp;- jTidy: jtidy-r938.jar: +externalLink{http://jtidy.sourceforge.net/license.html,MIT license}<br><br>
// Additionally, to be able to export images or DrawPanes and FacetCharts in IE6-8, the following libraries are needed:<br>
// &nbsp;&nbsp;- Batik: batik-anim, batik-awt-util, batik-bridge, batik-css, batik-dom, batik-ext,
// batik-gvt, batik-parser, batik-script, batik-svg-dom, batik-util, batik-xml:
// +externalLink{http://xmlgraphics.apache.org/batik/license.html,Apache License&#44; Version 2.0}<br>
// &nbsp;&nbsp;- xml-commons External Components: xml-apis-ext:
// +externalLink{http://www.w3.org/Consortium/Legal/2002/copyright-software-20021231,W3C Software Notice and License}
// </li>
// <p>
// <li><b>isomorphic_web_services</b>: Web services examples only.  Contains code backing the
// SmartClientOperations.wsdl example.  Do not deploy in production.<br>
// &nbsp;&nbsp;<u>Requires</u>:<br>
// &nbsp;&nbsp;&nbsp;&nbsp;isomorphic_core_rpc<br>
// &nbsp;&nbsp;&nbsp;&nbsp;axis<br>
// &nbsp;&nbsp;&nbsp;&nbsp;axis-schema<br>
// &nbsp;&nbsp;&nbsp;&nbsp;commons-discovery<br>
// &nbsp;&nbsp;&nbsp;&nbsp;commons-logging<br>
// &nbsp;&nbsp;&nbsp;&nbsp;jaxrpc<br>
// &nbsp;&nbsp;&nbsp;&nbsp;saaj<br>
// &nbsp;&nbsp;&nbsp;&nbsp;wsdl4j<br>
// </li>
// <li><b>isomorphic_tools</b>: Contains back-end logic for the "Admin Console" tool visible in
// the Developer Console, and also standalone from the SDK home page.  Also contains the the
// various data importers and exporters, and the server-side components of the BatchUploader.<br>
// &nbsp;&nbsp;<u>Requires</u>:<br>
// &nbsp;&nbsp;&nbsp;&nbsp;isomorphic_core_rpc<br>
// &nbsp;&nbsp;&nbsp;&nbsp;isomorphic_sql<br>
// </li>
// <li><b>isomorphic_embedded_tomcat</b>: This contains the bootstrap class for the Embedded
// Tomcat engine shipped with the SmartClient SDK.  It is not intended to be used in any way
// other than via the start_embedded_tomcat.bat|sh|command scripts in the webRoot of the
// SDK.<br>
// &nbsp;&nbsp;<u>Requires</u>:<br>
// &nbsp;&nbsp;&nbsp;&nbsp;isomorphic_core_rpc<br>
// &nbsp;&nbsp;&nbsp;&nbsp;All JARs under WEB-INF/embeddedTomcat/lib<br>
// </li>
// <li><b>isomorphic_spring</b>: Required for +link{dmiOverview,DMI} dispatches to Spring beans (via
// +link{serverObject.lookupStyle} : "spring").<br>
// &nbsp;&nbsp;<u>Requires</u>:<br>
// &nbsp;&nbsp;&nbsp;&nbsp;isomorphic_core_rpc<br>
// &nbsp;&nbsp;&nbsp;&nbsp;spring<br>
// &nbsp;&nbsp;&nbsp;&nbsp;commons-logging<br>
// </li>
// <li><b>isomorphic_examples</b>: Contains code for various SmartClient examples shown in the
// SDK.  Do not deploy to production.  The source code for the compiled classes in this module
// are present in the SDK in various locations and typically linked to directly from the
// examples that use them.<br>
// &nbsp;&nbsp;<u>Requires</u>:<br>
// &nbsp;&nbsp;&nbsp;&nbsp;isomorphic_core_rpc<br>
// &nbsp;&nbsp;&nbsp;&nbsp;isomorphic_sql<br>
// &nbsp;&nbsp;&nbsp;&nbsp;isomorphic_struts<br>
// &nbsp;&nbsp;&nbsp;&nbsp;isomorphic_spring<br>
// &nbsp;&nbsp;&nbsp;&nbsp;isomorphic_hibernate<br>
// &nbsp;&nbsp;&nbsp;&nbsp;commons-logging<br>
// &nbsp;&nbsp;&nbsp;&nbsp;jstl<br>
// &nbsp;&nbsp;&nbsp;&nbsp;jakarta-taglib-standard<br>
// </li>
// <li><b>isomorphic_realtime_messaging</b>: Server support required for the SmartClient
// Realtime Messaging Module.  Install this if you're using this 'push' technology.  For more
// information, see: +docTreeLink{Messaging QuickRef}:<br>
// &nbsp;&nbsp;<u>Requires</u>:<br>
// &nbsp;&nbsp;&nbsp;&nbsp;isomorphic_core_rpc<br>
// </li>
// <li><b>isomorphic_hibernate</b>: Contains support for Hibernate DataSources as described
// here: +link{group:hibernateIntegration}.<br>
// &nbsp;&nbsp;<u>Requires</u>:<br>
// &nbsp;&nbsp;&nbsp;&nbsp;isomorphic_core_rpc<br>
// &nbsp;&nbsp;&nbsp;&nbsp;isomorphic_sql<br> 
// &nbsp;&nbsp;&nbsp;&nbsp;hibernate3<br>
// &nbsp;&nbsp;&nbsp;&nbsp;commons-logging<br>
// &nbsp;&nbsp;&nbsp;&nbsp;dom4j<br>
// &nbsp;&nbsp;&nbsp;&nbsp;jdbc2_0-stdext<br>
// &nbsp;&nbsp;&nbsp;&nbsp;asm<br>
// &nbsp;&nbsp;&nbsp;&nbsp;asm-attrs<br>
// &nbsp;&nbsp;&nbsp;&nbsp;jta<br>
// &nbsp;&nbsp;&nbsp;&nbsp;cglib<br>
// &nbsp;&nbsp;&nbsp;&nbsp;antlr<br>
// &nbsp;&nbsp;<u>Optionally Requires</u>:<br>
// &nbsp;&nbsp;&nbsp;&nbsp;hsqldb - not required for linking, but needed for Hibernate to
// actually connect to databases of this type, and hsqldb is what's used in the SDK examples.
// Note that for production use you will need a JDBC driver for your database type.<br>
// </li>
// <li><b>isomorphic_jpa</b>: Contains support for JPA DataSources as described +link{group:jpaIntegration,here}.<br>
// &nbsp;&nbsp;<u>Requires</u>:<br>
// &nbsp;&nbsp;&nbsp;&nbsp;isomorphic_core_rpc<br>
// &nbsp;&nbsp;&nbsp;&nbsp;Hibernate and its dependencies or any other JPA persistence provider<br>
// </li>
// <li><b>isomorphic_sql</b>: The SmartClient SQLDataSource.<br>
// &nbsp;&nbsp;<u>Requires</u>:<br>
// &nbsp;&nbsp;&nbsp;&nbsp;isomorphic_core_rpc<br> 
// &nbsp;&nbsp;&nbsp;&nbsp;commons-dbcp<br> 
// &nbsp;&nbsp;&nbsp;&nbsp;hsqldb<br> 
// </li>
// </li>
// <li><b>isomorphic_autotest</b>: Support for 
// +link{group:automatedTesting,automated testing and Continuous Integration}<br>
// &nbsp;&nbsp;<u>Requires</u>:<br>
// &nbsp;&nbsp;&nbsp;&nbsp;isomorphic_core_rpc<br> 
// &nbsp;&nbsp;&nbsp;&nbsp;selenium-server<br>
// &nbsp;&nbsp;<em>The <code>selenium-server.jar</code> file provided with SmartClient is a 
// trimmed version that does not include all of the libraries used by Selenium itself.
// Therefore the following libraries are also needed:</em><br>
// &nbsp;&nbsp;&nbsp;&nbsp;backport-util-concurrent<br> 
// &nbsp;&nbsp;&nbsp;&nbsp;commons-configuration<br> 
// &nbsp;&nbsp;&nbsp;&nbsp;commons-exec<br> 
// &nbsp;&nbsp;&nbsp;&nbsp;commons-io<br> 
// &nbsp;&nbsp;&nbsp;&nbsp;guava<br> 
// &nbsp;&nbsp;<em><u>NOTE:</u> <code>servlet-api.jar</code> is also required if you intend to
// run Selenium RC from a standalone process (ie, from a normal Java program, not a webapp).  
// However, it should <u>not</u> be deployed to a servlet container such as Tomcat or Jetty.
// The best case is that the file will be unused and a source of confusion for anybody looking
// at the webapp's library usage; the worst case is that it will conflict with the container's 
// own implementation of the Servlets API.</em>
// </li>
// <li><b>isomorphic_js_parser</b>: A parser capable of reading a JSON byte stream and creating
// an in-memory Java object structure to match.  Used by any mechanism that relies on
// JSON-style configuration.  Examples include FileAssembly definitions in JSON format, any use
// of the rulesFile with a URIRegexFilter (Java Servlet) or subclass.  Otherwise, generally not
// required except as documented in another module (e.g., isomorphic_core_rpc).
// <br>
// &nbsp;&nbsp;<u>Requires</u>:<br>
// &nbsp;&nbsp;&nbsp;&nbsp;isomorphic_core_rpc<br> 
// </li>
// <li><b>isomorphic_compression</b>: This is a part of the Network Performance Module.  The
// isomorphic_compression module is required for dynamic and static compression of various
// assets delivered to the browser.  For more information, see: 
// +link{group:compression}.<br>
// &nbsp;&nbsp;<u>Requires</u>:<br>
// &nbsp;&nbsp;&nbsp;&nbsp;isomorphic_core_rpc<br> 
// </li>
// <li><b>isomorphic_assembly</b>: This is part of the Network Performance Module.  The
// isomorphic_assembly module is required for file assembly and stripping.  For more
// information, see: +link{group:fileAssembly}.<br>
// &nbsp;&nbsp;<u>Requires</u>:<br>
// &nbsp;&nbsp;&nbsp;&nbsp;isomorphic_core_rpc<br> 
// &nbsp;&nbsp;<u>Optionally Requires</u>:<br>
// &nbsp;&nbsp;&nbsp;&nbsp;isomorphic_obfuscation - for code obfuscation<br> 
// </li>
// <li><b>isomorphic_obfuscation</b>: This is part of the Network Performance Module.  The
// isomorphic_obfuscation module is required to enable obfuscation of code assembled by the
// File Assembly system.  For more information, see: 
// +link{group:compression}.<br>
// &nbsp;&nbsp;<u>Requires</u>:<br>
// &nbsp;&nbsp;&nbsp;&nbsp;isomorphic_core_rpc<br> 
// </li>
// <li><b>isomorphic_struts</b>: Contains the ValidationAction and ValidationFailureAction
// classes that implement RPC-based validation of DynamicForms using the Struts
// ValidatorPlugIn.  If you're not using Struts or if you don't need this particular feature of
// SmartClient, you do not need this module or its dependencies (also see the important note 
// below).  An example of this style of validation is available here: 
// +externalLink{/examples/struts/forms/welcome.do} - read the info on this page, and follow 
// the "Dynamic Form (With RPC-based Validation)" Link for the actual example.<br><br>
// <b>NOTE:</b> This support is for Struts 1.0 only, and is only intended to be used in certain
// edge cases of incremental migration to SmartClient.  You should only use it if directed to 
// do so by Isomorphic Support.<br><br>
// &nbsp;&nbsp;<u>Requires</u>:<br>
// &nbsp;&nbsp;&nbsp;&nbsp;isomorphic_core_rpc<br>
// &nbsp;&nbsp;&nbsp;&nbsp;struts<br>
// &nbsp;&nbsp;&nbsp;&nbsp;commons-digester<br>
// &nbsp;&nbsp;&nbsp;&nbsp;commons-beanutils<br>
// &nbsp;&nbsp;&nbsp;&nbsp;commons-fileupload<br>
// &nbsp;&nbsp;&nbsp;&nbsp;commons-logging<br>
// &nbsp;&nbsp;&nbsp;&nbsp;commons-validator<br>
// &nbsp;&nbsp;&nbsp;&nbsp;jakarta-oro<br>
// </li>
// </ul>
//
// @treeLocation Concepts/Deploying SmartClient
// @title Java Module Dependencies
// @visibility external
//<


//> @groupDef iscServer
//
// The SmartClient Server is a set of Java libraries, servlets and tools that provide the key
// server-side components needed to build a complete application in the modern web
// architecture.
// <P>
// The SmartClient Server can be +link{group:iscInstall,integrated} into any pre-existing Java
// application, and is designed to rapidly connect SmartClient visual components to
// pre-existing Java business logic or persistence engines.  SmartClient's Ajax request
// processing facilities can be easily integrated into Struts Actions,
// +link{group:springIntegration,Spring controllers}, or custom servlets and JSPs.
// <P>
// Alternatively, the SmartClient Server provides a complete SQL and Hibernate-based
// persistence engine for new applications, with out-of-the-box servlets for processing Ajax
// data requests.
// <P>
// The SmartClient Server is optional, and SmartClient's client-side Ajax engine can be
// integrated with any server that provides HTTP access, using XML, JSON, SOAP or proprietary
// data protocols.  However any server in a modern web application will be required to provide
// most or all of the features of the SmartClient Server (described below), and the SmartClient
// Server represents a best-of-breed implementation of these facilities with a long history of
// high-volume production deployments.
// <P>
// <h4>Server enforcement of +link{class:Validator,Validators}</h4>
// <P>
// Data passed from the browser can be automatically validated by the SmartClient Server.
// In contrast, when using +link{group:clientDataIntegration,client-side integration},
// data arrives as HTTP params or XML messages, and you must parse values into the correct
// types (eg java.util.Date) and validate them, or use a server framework that does so.
// <P>
// <h4>High Speed Data Delivery / Data Compression</h4>
// <P>
// The SmartClient Server delivers data to and from the browser using a proprietary, maximally
// efficient protocol, providing simple Java APIs for sending and receiving data.
// <P>
// SmartClient's data protocol is:
// <ul>
// <li> automatically compressed: provides 6-8x improvement in bandwidth utilization
// <li> efficient on the server: high speed data serialization for any Java Object
// <li> efficient in the browser: faster than ordinary XML or JSON data delivery
// <li> minimal: facilities for +link{dataSource.dropExtraFields,trimming} and
// +link{dataSourceField.valueXPath,extracting} only the data you want the browser to see
// </ul>
// <P>
// <h4>Transparent upload support</h4>
// <P>
// SmartClient provides special client and server-side support for +link{group:upload,file
// upload}, which allows single and multiple-file HTTP uploads to be performed as a background
// Ajax request without reloading the page or launching sub-windows.
// <P>
// Uploaded files arrive at the SmartClient server as Java InputStreams accessible from the
// DSRequest object, and can optionally be automatically stored via SmartClient's SQL
// subsystem.
// <P>
// <h4>Transparent Queuing / "Batch" Operations</h4>
// <P>
// Any request transmitted to the SmartClient Server can be combined into a "queue" transmitted
// as a single HTTP request, with in-order execution for all queued operations. 
// +link{RPCManager.startQueue,startQueue()} starts a queue and
// +link{RPCManager.sendQueue,sendQueue()} transmits it; queuing is transparent to the code
// that initiates the individual requests.  This enables:
// <ul>
// <li> re-use of data access operations across different screens
// <li> easy implementation of transaction boundaries
// <li> simplified saving and loading of screens with complex, master-detail views
// <li> guaranteed in-order processing of operations
// <li> more efficient network usage
// </ul>
// <P>
// <h4>Visual Builder</h4>
// <P>
// +link{group:visualBuilder,Visual Builder} is included with the SmartClient Server, and uses
// server features such as automatic SQL binding to provide a rapid prototyping environment.
// <P>
// <h4>Automatic Bi-directional Java &lt; - &gt; JavaScript serialization and translation</h4>
// <P>
// Provides a powerful, type-safe +link{rpcRequest.data,data transmission mechanism} for moving
// data between a Java server and the browser. 
// <P>
// Any Java objects, including Java Beans, POJOs, Java Collections, XML DOMs and all Java
// primitives, with any level of nesting, can be automatically serialized and delivered as
// JavaScript Objects to the SmartClient client-side components.
// <P>
// JavaScript Objects existing in the browser can likewise be automatically transmitted to a
// Java Server and translated to Java Objects, with any level of nesting and automatic
// preservation of primitive types.
// <P>
// <h4>SQL and Hibernate connectors</h4>
// <P>
// DataSources of serverType:"sql" or serverType:"hibernate" can generate and execute queries
// against popular SQL engines or against the Hibernate ORM system, providing SmartClient's
// +link{DataBoundComponent}s with the four standard CRUD operations (create, retrieve, update,
// delete) without writing any server-side code.  For rapid prototyping, these DataSources can
// even generate SQL tables based on the DataSource declaration, using the
// +link{group:adminConsole} visual tool.
// <P>
// Server-side APIs allow server-side modification of the request before it is executed (for
// example, to enforce security) and post-processing of the request after execution (for
// example, to provide calculated values).
// <P>
// Both serverType:"sql" and serverType:"hibernate" support the field-operator-value queries
// that can be generated by using the +link{FilterBuilder} component (see
// +explorerExample{filterBuilderBracket,example}).
// <P>
// <h4>Rich, Standardized Request / Response protocol</h4>
// <P>
// The SmartClient Server provides a standardized request and response protocol designed for
// data-oriented "CRUD" operations (create, retrieve, update, delete).
// <P>
// This standardized protocol automatically handles +link{DSRequest,request metadata} (paging
// parameters, requested sort order, original values of data being modified) and
// +link{DSResponse,response metadata} (error handling, cache management, session expiration etc).
// <P>
// Similar to the impact of Struts on early web applications, this standardized protocol avoids
// developers in different groups inventing their own incompatible and redundant
// request/response protocols, and allows developers to more easily learn code they didn't
// author.
// <P>
// <h4>Bi-directional XPath binding to Java Objects</h4>
// <P>
// Most UI designs do not directly reflect the underlying Object model and so some degree of
// translation is necessary in order to populate UI components with data and apply user changes
// to the Java Object model.  This is often accomplished with brittle, difficult to understand
// data translation code sprinkled throughout the system, done in a different way for every
// screen or component.
// <P>
// SmartClient provides a standard, +link{dataSourceField.valueXPath,XPath-based approach} to
// adapting any Java-based Object model to the requirements of the UI design.  Data relevant to
// the application UI is centrally extracted in the server-side +link{DataSource} layer, so
// that all UI components have a consistent, unified view of the data model for both loading
// <b>and</b> saving data.
// <P>
// <h4>Broadest possible browser support</h4>
// <P>
// The SmartClient Server can compensate for facilities 
// +link{group:platformDependencies,missing or disabled in certain browsers},
// including ActiveX being disabled in IE6 and missing XML support in some versions
// of Apple's Safari browser.
// <P>
// <h4>Transparent Proxying</h4>
// <P>
// +link{RPCManager.sendProxied,Proxying} allows SmartClient applications to access web
// services, RSS feeds, HTML content and other data services in a secure manner regardless of
// where they are located: across the enterprise or publicly available.
// <P>
// <h4>Optional +link{group:networkPerformance,Network Performance} Module</h4>
// <P>
// Provides:
// <ul>
// <li> compressed delivery of SmartClient runtime, application logic and other assets such as CSS
// <li> +link{FileLoader,background download} of SmartClient and other assets for zero
// user-perceived load time
// <li> on-the-fly stripping, combining and obfuscation of JavaScript (application code and data)
// <li> browser cache control
// </ul>
// <P>
// <h4>Optional Messaging Module (aka server push)</h4>
// <P>
// The +docTreeLink{messaging,Messaging} module allows the server to "push" messages to the client, without
// client polling, for real-time monitoring/dashboarding applications.
//
// @title SmartClient Server Summary
// @treeLocation Java Server Reference
// @visibility external
//<


//> @groupDef jsfIntegration
//
// SmartClient can be used within JSF applications to add AJAX richness and interactivity.
// <P>
// Because JSF is a pre-AJAX architecture, the recommended approach in adding
// SmartClient to JSF applications is to create pages that use SmartClient components exclusively,
// so that older, server-based JSF components do not introduce full-page refreshes.
// <P>
// JSF pages that render components on the server access data via JSF Expression Language.
// SmartClient-based JSF pages can similarly load initial data by using JSTL, as shown in 
// +externalLink{/examples/server_integration/#jstlList,this example}, where a ListGrid is
// populated by JSTL access to Java Beans stored in the JSP <code>pageContext</code>.  
// <P>
// Once a SmartClient JSF page has loaded, SmartClient components will request data via background
// HTTP requests that load only data, not a complete page.  The 
// +link{dmiOverview,Direct Method Invocation} system can be used to declaratively map SmartClient's
// background data requests directly to Java Methods. The SmartClient server automatically
// translates inbound request data into Java Objects that are passed to the method you specify,
// and the Java method return value is automatically translated into data for SmartClient
// components.
// <P>
// <h4>Incorporating server-side JSF components into a SmartClient JSF page</h4>
// <P>
// An +link{HTMLFlow} or +link{HTMLPane} component can be used to incorporate server-generated
// content within a SmartClient-based page.  With +link{htmlFlow.contentsType,contentsType} set to
// "page", the HTMLPane/Flow will act like a standalone page-within-a-page (via a
// SmartClient-managed HTML IFRAME element), allowing interactive server-side JSF components to
// participate normally, with limitations as discussed under the documentation for 
// +link{htmlFlow.contentsType,contentsType}.
//
// @treeLocation Concepts
// @title Integration with JSF
// @visibility external
//<

//> @groupDef strutsIntegration
//
// <b>Overview of SmartClient integration with Struts and other popular server-side
// frameworks.</b>
// <p>
// Current and upcoming server-side frameworks solve a lot of problems endemic to the past
// generation of web applications.  Piles of JSPs and Servlets have been replaced by MVC and
// other paradigms that enhance developer productivity.  Unfortunately the end-user
// presentation remains vanilla HTML.  SmartClient solves this problem by providing rich
// databindable components. SmartClient was designed to integrate cleanly with existing
// server-side frameworks.  Because SmartClient components only require an HTML context to
// render, they can be used with any server-side frameworks that use HTML for presentation.
// <p>
// SmartClient provides a rich UI by doing more work on the client (browser) rather than
// sending every user action to the server for re-rendering.  Instead of doing page transitions
// to redraw the whole page, SmartClient sends RPC (Remote Procedure Call) requests (or AJAX
// requests) to the server while the UI allows the user to continue interacting with the
// system.  Current server-side frameworks, on the other hand are typically designed around
// page transitions - for example in Struts user actions are typically mapped to URLs that
// dispatch through a central servlet and ultimately return new HTML to be rendered by the
// browser.  The problem with page transitions is that they destroy client-side state and
// introduce client-server latency (and generally use more bandwidth since HTML is sent over
// the wire rather than just data) - essentially destroying a large part of the Rich
// Internet Application (RIA) experience.
// <p>
// Fortunately, there's a way to get the best of both worlds - to leverage the power of your
// favorite server-side framework and combine it with the SmartClient UI.  There are several
// approaches to integrating SmartClient into an existing framework:
// <p>
// <u><b>Plug-replacing HTML components with SmartClient components</b></u>
// <p>
// SmartClient components can be instructed to draw at page load time using by specifying
// <code>position: "relative"</code> at construction time.  This enables you to replace any
// chunk of HTML with a SmartClient component - the new component simply inserts its HTML in
// the page flow during page load.  This is the easiest integration option - you get a better
// UI with minimal work.  The downside is that you don't get the full power of a rich client
// because most user actions will still trigger a page transition.
// <p>
// <u><b>Eliminating page transitions</b></u>
// <p>
// Most SmartClient components can accept new data (or even dynamically pre-fetch and expire
// data) without needing to be recreated.  For example - let's say you want to draw a grid on a
// page.  In a traditional server-side-rendered application the server would generate all of
// the html with "next 20 records" and "previous 20 records" buttons. When the user wants to
// see the next set of data, he clicks one of the buttons and the server replaces the entire
// page with a new grid that contains the next/previous 20 records.  In a SmartClient
// application, you would create a databound ListGrid.  Based on its configuration this grid
// will fetch the first N (say 20) records and display a scrollbar for the user to scroll
// through the data.  When the user scrolls off the last cached record the ListGrid
// automatically sends an RPC to the server asking for the next 20 records.  This RPC (fetch)
// is performed without destroying the page the user is currently looking at - it just happens
// seamlessly in the background.  If the user now scrolls back to the first 20 records -
// they're already cached in the grid, so no fetch is performed.  Of course, in a real world
// application, it's typical that a page has hundreds of components and in a server-side-only
// rendering all of them need to be rebuilt by the server and resent to the client when a piece
// of data in just one needs to be updated.  SmartClient components can intelligently update
// just their data without the need to redraw the whole page.
// <p>
// The plug-replacement strategy listed above gives us a SmartClient component in place of a
// raw HTML rendering.  Now we need to databind that component so that actions like scrolling a
// grid or validating a form don't cause a page transition.  To do this, you need to set up a 
// Struts Action that will handle <b>all</b> SmartClient <code>RPCRequest</code>s and 
// <code>DSRequest</code>s.  This is important, as requests need to be sent to the same URL 
// to enable queuing to work.  In your Action class, you simply need to invoke 
// <code>RPCManager.processRequest()</code> to hook straight into the normal 
// <code>DSRequest</code> processing flow.
// <p>
// The SDK contains a simple example of doing form validation without incurring a page
// transition. These examples also show how to populate e.g. field names using the struts-bean
// taglib and how to set validation errors using the standard Struts Validation plugin.
// Point your browser to +externalLink{/examples/struts/forms} in the SmartClient SDK to take a
// look.
//
// @treeLocation Concepts
// @title Integration with Struts
// @visibility external
//<

//> @groupDef springIntegration
// <b>Overview</b>
// <P>
// The Spring framework has many different parts, from integration with Object Relational
// Mapping (ORM) and transaction management systems, to a Model View Controller (MVC)
// architecture.
// <P>
// If you are building a new application from scratch and/or you are trying to 
// modernize the presentation layer of an existing application, most of Spring MVC is
// inapplicable in the +link{group:smartArchitecture,SmartClient architecture}.  Specifically,
// SmartClient renders <b>all</b> HTML on the client, and the server is responsible only for
// retrieving data and enforcing business rules.  This means that Spring's ModelAndView and all
// functionality related to retrieving and rendering Views is unnecessary in SmartClient.
// SmartClient only needs the Model, and provides methods to deliver that Model to SmartClient
// components (the server side method DSResponse.setData()).
// <P>
// However, Spring's DispatchServlet, Handler chain, and Controller architecture is applicable
// to SmartClient.  See "Using Spring Controllers" below.
// <P>
// <b>Existing Spring Application</b>
// <P>
// As discussed under the general +link{group:clientServerIntegration,server integration}
// topic, integrating SmartClient into your application involves finding a way to provide data
// that fulfills the +link{DSRequest,DataSource requests} sent by SmartClient components.
// <P>
// There are 2 approaches for integrating SmartClient into an existing Spring application:
// <ul>
// <li> <b>call Spring beans via SmartClient DMI</b> <span
// style="color:red">[Recommended]</span>: use SmartClient Direct Method Invocation
// (DMI) to map +link{dsRequest,DataSource requests} to beans managed by Spring, via
// +link{serverObject.lookupStyle}:"spring".   Return data to the browser by either simply
// returning it from your method, or via creating a DSResponse and calling DSResponse.setData()
// (server-side method). 
// <P>
// This is the easiest method and produces the best result.  A Collection of Java Beans, such
// as EJB or Hibernate-managed beans, can be directly returned to SmartClient as the result of
// a DMI method, without the need to create an intervening
// +externalLink{http://en.wikipedia.org/wiki/Data_transfer_object,Data Transfer Object} to express
// which fields should be delivered to the browser - instead, only the fields declared on the
// DataSource are returned to the browser (see
// +link{DataSource.dropExtraFields,dropExtraFields}.  In this integration scenario, the
// majority of the features of the SmartClient Server framework still apply - see this
// +link{group:featuresCustomPersistence,overview}.
// <P>
// <li> <b>configure Spring to return XML or JSON responses</b>: create variants
// on existing Spring workflows that use a different type of View in order to output XML or
// JSON data instead of complete HTML pages.  The SmartClient +link{RestDataSource} provides a
// standard "REST" XML or JSON-based protocol you can implement, or you can adapt generic
// +link{DataSource,DataSources} to existing formats.
// <P>
// In some Spring applications, all existing Spring workflows can be made callable by
// SmartClient with a generic View class capable of serializing the Model to XML or JSON,
// combined with a Controller that always uses this View.  Consider the following Java
// anonymous class, which uses the SmartClient JSTranslater class to dump the entire 
// Spring Model as a JSON response.
// <pre>
//  new View() {
//        public void render(Map model, HttpServletRequest request,
//                           HttpServletResponse response) throws IOException {
//                final ServletOutputStream outputStream = response.getOutputStream();
//                response.setContentType("application/x-javascript");
//                outputStream.println(JSTranslater.get().toJS(model));
//                outputStream.close();
//        }
//        public String getContentType() {
//                return "application/x-javascript";
//        }
//  }
// </pre>
// <P>
// If you use this approach, you do not need to install the SmartClient server, and can
// +link{iscInstall,deploy} SmartClient as simple web content (JS/media/HTML files).  If you
// are already familiar with how to generate XML from objects that typically appear in your
// Spring Models, this may be the easiest path.
// </ul>
// <P>
// <b>Using Spring Controllers with SmartClient DMI</b>
// <P>
// You can create a Controller that invokes standard SmartClient server request processing,
// including DMI, like so:
// <pre>
// public class SmartClientRPCController extends AbstractController
// {
//     public ModelAndView handleRequest(HttpServletRequest request, 
//                                       HttpServletResponse response)
//         throws Exception
//     {
//         // invoke SmartClient server standard request processing
//         com.isomorphic.rpc.RPCManager.processRequest(request, response);
//         return null; // avoid default rendering
//     }
// }
// </pre>
// This lets you use Spring's DispatchServlet, Handler chain and Controller architecture as a
// pre- and post-processing model wrapped around SmartClient DMI.
//
// @treeLocation Concepts
// @title Integration with Spring
// @visibility external
//<



//> @groupDef hibernateIntegration
//
// SmartClient can integrate with Hibernate in two main ways, both of which are enabled by
// creating a DataSource descriptor (.ds.xml file) with
// +link{DataSource.serverType,serverType="hibernate"}:
// <ul>
// <li> Pre-existing beans: a SmartClient DataSource can be automatically derived from either a
// Hibernate-managed bean or the Hibernate mapping.  Use
// +link{dataSource.schemaBean,schemaBean} to derive from the bean or
// +link{DataSource.autoDeriveSchema} to derive from the mapping.  In this case you will
// initially have a very short .ds.xml per bean - no &lt;fields&gt; are required unless
// and until you want to override the automatically derived fields.
// <li> "Beanless" mode: SmartClient can drive Hibernate as a storage layer only, automatically
// generating Hibernate configuration from a SmartClient DataSource file
// (<i>dataSourceID</i>.ds.xml).  In this case, you do not write a Java bean or create
// Hibernate mappings; Hibernate's beanless
// +externalLink{http://www.hibernate.org/hib_docs/v3/reference/en/html_single/#persistent-classes-dynamicmodels,"dynamic model"}
// mode is used.
// </ul>
// <P>
// Which mode to use is primarily a matter of preference and pre-existing code.  However, if
// you do not have pre-existing code or other special circumstances, the following approach is
// the most productive:
// <ol> 
// <li> use "beanless" mode, specifying fields in SmartClient's .ds.xml format (far more
// compact than a Java bean with a getter and setter for each field)
// <li> add business logic as needed via DMI, Server Scripting, custom server validators, and
// other approaches covered in the QuickStart Guide
// <li> call any reusable DMI methods both via your SmartClient UI and via other, non-UI
// related Java logic (the DMI methods are now a reusable "data services tier")
// <li> only create an actual Java bean if you discover re-usable, bean-specific business
// logic that cannot be encapsulated as a data service (rare)
// </ol>
// <P>
// Note that the +link{group:adminConsole,Admin Console}'s
// "Import DataSources" section can be used to import test data into serverType:"hibernate"
// DataSources in the same manner as SQLDataSources.
// <p>
// HibernateDataSource supports operations with composite primary keys. Setting data source level property
// +link{attr:dataSource.idClassName,idClassName} to fully qualified class name indicates,
// that entity uses composite primary key.
// <p>
// <h3>Hibernate relations</h3>
// <p>
// For Hibernate integration where Java beans have been explicitly declared,
// HibernateDataSource supports automatic handling of Hibernate relations that don't declare a
// concrete field to hold ID values - see +link{group:jpaHibernateRelations}.
// <p>
// <h3>Hibernate Configuration</h3>
// <p>
// You can provide Hibernate configuration to the SmartClient server in three ways:
// <ul>
// <li>You can place a traditional <code>hibernate.cfg.xml</code> file somewhere on the 
//     classpath</li>
// <li>You can have SmartClient look up a Hibernate <code>Configuration</code> to use.  This 
//     works in the same way as a +link{ServerObject}, and in fact makes use of the 
//     ServerObject code, though note that lookupStyle "attribute" is not supported.  To look 
//     up a configuration, add ServerObject-compliant properties to your 
//     +link{server_properties,server.properties} file, prefixed with <code>hibernate.config</code>.  For
//     example: <pre>
//        hibernate.config.lookupStyle: spring
//        hibernate.config.bean: mySessionFactory
// </pre></li>
// <li>You can provide a Hibernate configuration at the level of individual DataSources, by 
//     specifying a +link{DataSource.configBean,configBean} on the dataSource (this is only 
//     applicable if you are using Spring; see below)</li>
// </ul>
// If you choose to have SmartClient lookup the Hibernate configuration, and you specify a 
// +link{serverObject.lookupStyle,lookupStyle} of "spring", SmartClient will make use of a
// Hibernate <code>SessionFactory</code> configured by Spring.  It is possible to set up multiple
// Hibernate configurations in Spring, and to map individual DataSources to different 
// configurations by making use of the <code>dataSource.configBean</code> property mentioned
// above.  Please note the following caveats:
// <ul>
// <li>DataSource-level Hibernate configuration is intended for unusual cases, such as when the
//     physical data store for one DataSource is actually a different database.  Hibernate 
//     relations between entities with different configurations do not work</li>
// <li>If you choose to configure Hibernate via Spring, "beanless" on-the-fly 
//     mappings are not supported; all entities must be hand-mapped to a bean, either in the 
//     properties of the Spring bean providing the configuration, in a <code>.cfg.xml</code>
//     file named in the Spring bean's <code>configLocation</code> property, or by use of 
//     persistence annotations in the actual mapped beans themselves</li>
// </ul>
// <p>
// <h3>Manual Hibernate Integration</h3>
// <p>
// In some cases you may not be able to immediately use the built-in HibernateDataSource - in
// this case take a look at +link{group:manualJpaHibernate,manual Hibernate integration}.
//
// @see attr:dataSource.beanClassName
// @see group:sqlConnectionPooling
// @treeLocation Concepts/Persistence Technologies
// @title Integration with Hibernate
// @visibility external
//<

//> @groupDef jpaIntegration
// To use JPA, set serverType="jpa" in your .ds.xml file, then set
// +link{attr:dataSource.beanClassName,beanClassName} to the fully qualified class name of the
// JPA entity.  For example:
// <pre>
// &lt;DataSource
//     ID="countryDS"
//     serverType="jpa"
//     beanClassName="com.smartgwt.sample.showcase.server.jpa.Country"
// &gt;
//     &lt;fields&gt;
// &lt;!-- ... Fields definition ... --&gt;
//     &lt;/fields&gt;
// &lt;/DataSource&gt;
// </pre>
// +link{DataSource.autoDeriveSchema} is supported for deriving DataSource fields from JPA
// entities automatically (except with JPA 1.0; see below).
// <p>
// Full support is provided for executing simple +link{Criteria}, with +link{AdvancedCriteria}
// supported if you have Power Edition or above.  However, note that there are limitations with case
// sensitive search in MySQL since MySQL automatically uses the 'like' operator in a
// case-insensitive manner and JPA does not correct this.  See <a
// href="http://dev.mysql.com/doc/refman/5.5/en/case-sensitivity.html">MySQL Reference Manual
// :: C.5.5.1 Case Sensitivity in String Searches</a> for more information.<p/>
// <p>
// If create a custom DataSource based on the built-in JPA functionality, subclass
// <code>com.isomorphic.jpa.JPA2DataSource</code>.
// <p>
// <h3>JPA configuration</h3>
// <p>
// JPA configuration should be specified in the <code>persistence.xml</code> file as usual, and placed
// in the <code>/WEB-INF/classes/META-INF</code> directory.  For JPA 2.0 make sure you correctly
// declare its usage in <code>persistence.xml</code>:<pre>
// &lt;persistence
//     version="2.0"
//     xmlns="http://java.sun.com/xml/ns/persistence"
//     xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
//     xsi:schemaLocation="http://java.sun.com/xml/ns/persistence 
//                         http://java.sun.com/xml/ns/persistence/persistence_2_0.xsd"
// &gt;
// ...
// &lt;/persistence&gt;</pre><br/>
// <P>
// JPADataSource supports operations with composite primary keys. Setting data source level property
// +link{attr:dataSource.idClassName,idClassName} to fully qualified class name indicates,
// that entity uses composite primary key.
// <P>
// JPADataSource supports automatic handling of JPA relations that don't declare a concrete
// field to hold ID values - see +link{group:jpaHibernateRelations}.
// <P>
// <h3>JPA 1.0 compatibility</h3>
// <P>
// To use JPA 1.0, set serverType="jpa1" instead.  JPA 1.0 does not support
// +link{dataSource.autoDeriveSchema}.  For JPA 1.0, the concrete implementation class (for subclassing to
// create a custom DataSource) is <code>com.isomorphic.jpa.JPADataSource</code>.
// <p>
// <h3>JPA transactions</h3>
// <p>
// JPA provides three mechanisms for transactions: for JEE applications JPA provides integration
// with JTA (Bean Managed Transactions and Container Managed Transactions); for JSE applications JPA has a native
// <code>EntityTransaction</code> implementation (Locally Managed Transactions). Spring framework is another popular
// way for declaring transactions in application.
// The transaction mechanism should be configured in the +link{server_properties,server.properties}
// file by setting
// property <b><code>jpa.emfProvider</code></b> to the fully qualified class name of the provider
// (implementation of <code>com.isomorphic.jpa.EMFProviderInterface</code>). SmartClient comes
// with five implementations:<ul>
// <li><b><code>com.isomorphic.jpa.EMFProviderLMT</code></b> - for Locally Managed Transactions.
//      Every fetch or DML operation starts a new transaction and commits after successful
//      execution.<br/>
//      This implementation reads the <b><code>jpa.persistenceUnitName</code></b> property from
//      the +link{server_properties,server.properties} file.  The value of this property needs
//      to be set to
//      the name of the persistence unit configured in <code>persistence.xml</code> file. For example:<pre>
// jpa.persistenceUnitName: PERSISTENCE_UNIT_NAME
//      </pre></li>
// <li><b><code>com.isomorphic.jpa.EMFProviderBMT</code></b> - for Bean Managed Transactions.
//      Every fetch or DML operation acquires the transaction object from the container and starts it.<br/>
//      This implementation reads two properties from the +link{server_properties,server.properties}
//      file:
//      <b><code>jpa.entityManager</code></b> and <b><code>jpa.entityManagerFactory</code></b>
//      containing appropriate resource name references configured in
//      <code>/WEB-INF/web.xml</code>. Configuration example:<pre>
// &lt;!-- EntityManager resource reference name declaration --&gt;
// &lt;persistence-context-ref&gt;
//    &lt;persistence-context-ref-name&gt;persistence/em&lt;/persistence-context-ref-name&gt;
//    &lt;persistence-unit-name&gt;PERSISTENCE_UNIT_NAME&lt;/persistence-unit-name&gt;
// &lt;/persistence-context-ref&gt;
//
// &lt;!-- EntityManagerFactory resource reference name declaration --&gt;
//  &lt;persistence-unit-ref&gt;
//      &lt;persistence-unit-ref-name&gt;persistence/emf&lt;/persistence-unit-ref-name&gt;
//      &lt;persistence-unit-name&gt;PERSISTENCE_UNIT_NAME&lt;/persistence-unit-name&gt;
//  &lt;/persistence-unit-ref&gt;
//
// #Property values for sample references:
// jpa.entityManager: persistence/em
// jpa.entityManagerFactory: persistence/emf
//      </pre></li>
// <li><b><code>com.isomorphic.jpa.EMFProviderCMT</code></b> - for Container Managed Transactions.
//      Every fetch or DML operation acquires the transaction object from the JEE container.
//      After successful method execution the container commits the transaction. In case of execution
//      failure <code>tx.setRollbackOnly()</code> is used to notify container to rollback the
//      transaction.<br/>
//      This implementation reads two properties from the +link{server_properties,server.properties}
//      file:
//      <b><code>jpa.entityManager</code></b> and <b><code>jpa.entityManagerFactory</code></b>
//      containing appropriate resource name references configured in
//      <code>/META-INF/ejb-jar.xml</code>. Configuration example:<pre>
// &lt;?xml version="1.0" encoding="UTF-8"?&gt;
// &lt;ejb-jar
//      version = "3.0"
//      xmlns = "http://java.sun.com/xml/ns/javaee"
//      xmlns:xsi = "http://www.w3.org/2001/XMLSchema-instance"
//      xsi:schemaLocation = "http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/ejb-jar_3_0.xsd"&gt;
//      &lt;enterprise-beans&gt;
//          &lt;session&gt;
//              &lt;ejb-name&gt;TestEJB&lt;/ejb-name&gt;
//              &lt;persistence-context-ref&gt;
//                  &lt;persistence-context-ref-name&gt;persistence/em&lt;/persistence-context-ref-name&gt;
//                  &lt;persistence-unit-name&gt;PERSISTENCE_UNIT_NAME&lt;/persistence-unit-name&gt;
//              &lt;/persistence-context-ref&gt;
//              &lt;persistence-unit-ref&gt;
//                  &lt;persistence-unit-ref-name&gt;persistence/emf&lt;/persistence-unit-ref-name&gt;
//                  &lt;persistence-unit-name&gt;PERSISTENCE_UNIT_NAME&lt;/persistence-unit-name&gt;
//              &lt;/persistence-unit-ref&gt;
//         &lt;/session&gt;
//     &lt;/enterprise-beans&gt;
// &lt;/ejb-jar&gt;
//
// #Property values for sample references:
// jpa.entityManager: persistence/em
// jpa.entityManagerFactory: persistence/emf
//      </pre></li>
// <li><b><code>com.isomorphic.jpa.EMFProviderNoTransactions</code></b> - transactions are
//      not used.<br/>
//      From the +link{server_properties,server.properties} file this implementation reads the
//      <b><code>jpa.persistenceUnitName</code></b> property which must containt the name of persistence unit
//      configured in <code>persistence.xml</code> file. For example:<pre>
// jpa.persistenceUnitName: PERSISTENCE_UNIT_NAME
//      </pre></li>
// <li><b><code>com.isomorphic.jpa.EMFProviderSpring</code></b> - for Spring Framework managed Transactions.
//      Every fetch or DML operation acquires the transaction object from the Spring Application Context.<br/>
//      This implementation reads two properties from the +link{server_properties,server.properties}
//      file:
//      <b><code>jpa.entityManagerFactory</code></b> and <b><code>jpa.transaction</code></b>
//      containing appropriate bean names configured in Spring Application Context.
//      You have to declare additional bean in your Spring Application Context to allow SmartClient
//      to acquire reference to context.
//      Configuration example:<pre>
// &lt;!-- SpringApplicationContextProvider bean definition required to get access to application context. --&gt;
// &lt;bean id="springApplicationContextProvider" class="com.isomorphic.spring.SpringApplicationContextProvider" /&gt;
//
// &lt;!-- Connection to data base --&gt;
// &lt;bean id="dataSource"
//      class="org.springframework.jdbc.datasource.DriverManagerDataSource"
//      p:driverClassName="DRIVER_CLASS"
//      p:url="CONNECTION_URL"
//      p:username="DB_USER_NAME"
//      p:password="DB_USER_PASSWORD" /&gt;
//
// &lt;!-- Reference to JPA EntityManagerFactory --&gt;
// &lt;bean id="entityManagerFactory" class="org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean"&gt;
//     &lt;property name="dataSource" ref="dataSource" /&gt;
//     &lt;property name="jpaVendorAdapter"&gt;
//        &lt;bean class="org.springframework.orm.jpa.vendor.HibernateJpaVendorAdapter"&gt;
//            &lt;property name="database" value="DB_TYPE" /&gt;
//        &lt;/bean&gt;
//     &lt;/property&gt;
//     &lt;property name="persistenceUnitName" value="PERSISTENCE_UNIT_NAME" /&gt;
// &lt;/bean&gt;
//
// &lt;!-- Reference to JpaTransactionManager --&gt;
// &lt;bean id="transactionManager" class="org.springframework.orm.jpa.JpaTransactionManager"&gt;
//     &lt;property name="entityManagerFactory" ref="entityManagerFactory" /&gt;
// &lt;/bean&gt;
//
// #Property values for sample bean names:
// jpa.entityManagerFactory: entityManagerFactory
// jpa.transaction: transactionManager
//      </pre></li>
// </ul>
// You can set <b><code>jpa.emfProvider</code></b> to your own implementation of
// <code>com.isomorphic.jpa.EMFProviderInterface</code> if you have specific requirements for
// transaction handling. <code>EMF</code> will instantiate provided implementation on initialization (static) and
// will use same instance every time. By using own implementation you can have complete control over creating/using
// <code>EntityManagerFactory</code> and <code>EntityManager</code> instances.<p/>
// <b>Additional configurations:</b><p/>
// In case you have several persistence units defined in your <code>persistence.xml</code> you can have additional
// configurations in +link{server_properties,server.properties} file. Additional configurations
// (prefixed with <b><code>jpa.</code></b>) should have name, <b><code>emfProvider</code></b> property and other
// properties required by specified EMF provider implementation.
// For example:<pre>
// jpa.configName.emfProvider: com.isomorphic.jpa.EMFProviderLMT
// jpa.configName.persistenceUnitName: ANOTHER_PERSISTENCE_UNIT_NAME</pre>
// To use additional JPA configuration you have to set <b><code>jpaConfig</code></b> property in data source
// definition:<pre>
// &lt;DataSource
//     ID="countryDS"
//     serverType="jpa"
//     beanClassName="com.smartgwt.sample.showcase.server.jpa.Country"
//     jpaConfig="configName"
// &gt;</pre>
// <b>Transaction management:</b>
// <p/><ul>
// <li>Operating under +link{RPCManager} (<code>+link{DSRequest}</code> has reference to
//   <code>+link{RPCManager}</code>):<ul>
//      <li>If participating in automatic transactions:<ul>
//          <li>retrieves existing transaction from <code>+link{RPCManager}</code> (if available);</li>
//          <li>starts new transaction (if not found in <code>+link{RPCManager}</code>);</li></ul></li>
//      <li>If one transaction per operation is used - starts new transaction;</li>
//      <li>Registers itself to <code>DSRequest.registerCallback()</code> for <code>onSuccess()</code>/
//      <code>onFailure()</code> execution to commit/roll back transaction;</li>
//      <li>Sets <code>DSRequest.setFreeOnExecute()</code> to <code>false</code> to postpone releasing
//          of <code>EntityManager</code> avoiding lazy loading exceptions when creating JS response and
//          traversing through persistent object tree;</li>
//      <li>Registers itself to <code>RPCManager.registerFreeResourcesHandler()</code> for <code>freeResources()</code>
//      execution to release <code>EntityManager</code>.</li></ul><br/>
// If you want to use same <code>EntityManager</code> and transaction in your custom data source implementation you can
// acquire it by <pre>JPAConnectionHolder holder = DataSource.getTransactionObject(req, EMF.TRANSACTION_ATTR);</pre>
// <code>JPAConnectionHolder</code> instance contains references to entity manager and transaction object
// used by <code>JPADataSource</code>. You should never commit/rollback automatic transaction.
// Overall commit/rollback will be issued by <code>RPCManager</code> and will be handled by <code>JPADataSource</code>
// object which started transaction.
// </li>
// <li>Operating without +link{RPCManager}:<ul>
//      <li>starts new transaction;</li>
//      <li>commits/rolls back transaction and releases <code>EntityManager</code> if
//          <code>DSRequest.setFreeOnExecute()</code> is set to <code>true</code> (defalut);</li>
//      <li>relies on calling code to call <code>onSuccess()</code>/<code>onFailure()</code> to commit/roll back
//          transaction and to call <code>freeResources()</code> to release <code>EntityManager</code>.<br/>
//          Example code for data source operation execution with manual transaction handling:
//          <pre>
//              DSRequest req = new DSRequest("myDS", "fetch");
//              req.setFreeOnExecute(false);
//              DSResponse resp = req.execute();
//              List dataList = resp.getDataList();
//              //... traverse through persistent object tree
//              // Commit current transaction.
//              ((JPADataSource) r.getDataSource()).onSuccess();
//              // Release entity manager.
//              ((JPADataSource) r.getDataSource()).freeResources(req);
//          </pre></li></ul>
// </li></ul>
// <p>
// <h3>Manual JPA Integration</h3>
// <p>
// In some cases you may not be able to immediately use the built-in JPADataSource - in
// this case take a look at +link{group:manualJpaHibernate,manual Hibernate integration}.
//
// @see group:sqlConnectionPooling
// @treeLocation Concepts/Persistence Technologies
// @title Integration with JPA
// @visibility external
//<

//> @groupDef jpaHibernateRelations
// JPA and Hibernate allow relations to be declared between entities where there is no actual
// Java field for storing the ID of a related entity, even though such a column exists in the
// database.  For example:
// <pre>
//    &#64;ManyToOne
//    &#64;JoinColumn(name="countryId", referencedColumnName="countryId")
//    private Country country;
// </pre>
// JPADataSource and HibernateDataSource support this style of declaration and will
// automatically handle mapping between IDs and entities.
// <p>
// The example above and the following examples assume a DataSource "country" for a
// JPA/Hibernate entity "Country" with an Id field of "countryId", and a DataSource "city" for a
// JPA/Hibernate entity "City" with an Id field of "cityId".
// <p>
// <h3>Many-To-One Relations</h3>
// <p>
// An example of Many-To-One is that Many "City"s belong to One "Country".  In Java, each City
// bean has a field of type Country.  In the database, rows for cities and countries are linked
// by ID.
// <p>
// To specify a many-to-one relation, declare a DataSourceField named after the Java field that
// declares the relation ("country" above) with the property
// +link{attr:DataSourceField.foreignKey,foreignKey} pointing to related DataSource's primary key:
// <pre>
//    &lt;field name="country" foreignKey="country.countryId"/&gt;
// </pre>
// When delivered to the browser, the value of the <code>country</code> field will be the ID of
// the related Country entity.  The <code>country</code> field can be treated as a normal text
// or integer field value, for example, you can use a +link{SelectItem} that uses
// +link{SelectItem.optionDataSource} to allow selecting the ID of a different related
// entity.  Then, when the new ID is saved to the server, JPADataSource automatically looks up
// the related object and persists the new relation to JPA.
// <p>
// <b>NOTE:</b>: do not declare a "type" attribute for such a field - these fields provide
// specialized mapping between IDs and JPA/Hibernate entities, so don't really have a single
// concrete type.
// <p>
// If you want fields from a related entity to be included whenever your entity is fetched, for
// example, whenever a city is fetched you want the <code>countryName</code> fetched from the
// related <code>country</code> entity, use +link{DataSourceField.includeFrom}.
// <p>
// <b>Automatic Criteria translation</b>
// <p>
// If criteria are submitted for a ManyToOne relation field containing an ID value, this will
// correctly return Records that are associated with the related object that has that ID.
// <p>
// For example, given a countryId of "1", you can fetch all city Records that are related to
// that countryId as follows:
// <var class="smartclient">
// <pre>
//   isc.DataSource.get("city").fetchData({ countryId: 1 }, <callback>);
// </pre>
// </var>
// <var class="smartgwt">
// <pre>
//    DataSource.get("city").fetchData(new Criteria("countryId", "1"), <callback>);
// </pre>
// </var>
// <p>
// <h3>One-to-Many Relations</h3>
// <p>
// An example of One-To-Many relation is that One "Country" has Many "City"'s.  Each "Country"
// has a list of cities within it, which may be declared as a Java bean property of Collection
// type (or List, Set, etc).
// <p>
// To specify a one-to-many relation, declare a DataSourceField that:
// <ul>
// <li> is named after the Java field that declares the OneToMany relation (whose type is a
//      Collection of the related entities)
// <li> declares its "type" to be the ID of the related DataSource
// <li> declares a +link{attr:DataSourceField.foreignKey,foreignKey} pointing to the related
//      DataSource's primaryKey field
// <li> sets multiple="true"
// </ul>
// For example, for a Country bean that has a Collection of City beans:
// <pre>
//     &lt;field name="cities" type="city" multiple="true" foreignKey="city.cityId"/&gt;
// </pre>
// With this declaration, whenever Records are loaded from the Country DataSource, they will
// contain a list of City Records as subobjects, accessible via 
// <var class="smartgwt"><code>countryRecord.getAttributeAsRecordList("cities")</code></var>
// <var class="client">countryRecord.cities</var>
// <p>
// If loading all related "city" records is desirable in some circumstances and not others, you
// can use +link{operationBinding.outputs} to avoid loading "city" records for certain
// operations.
// <p>
// <h3>Alternative: Many-To-One loading complete related object</h3>
// <p>
// For a Many-To-One relation, instead of loading just the ID of the related object, you can
// load the entire related object as a nested Record.  To do so, just declare the type of the
// field to be the ID of the related DataSource, rather than leaving type unset:
// <pre>
//     &lt;field name="country" type="country" foreignKey="country.countryId"/&gt;
// </pre>
// The nested "country" Record will be available on a "city" record via 
// <var class="smartgwt"><code>cityRecord.getAttributeAsRecord("country")</code></var>
// <var class="client">cityRecord.country</var>.
// Saving a City record that contains a nested Country record in the "country"
// attribute will result in the Country being updated in JPA/Hibernate.
// <P>
// This mode is not typically used, since loading just the ID of the related Country object is
// more efficient if many cities are being loaded, and the related Country object can always be
// loaded with a second fetchData() data, which can still be done in a single HTTP request via
// +link{isc.RPCManager.startQueue(),queuing}.
// <P>
// <h3>Alternative: One-To-Many loading related IDs</h3>
// <p>
// For a One-To-Many relation, instead of loading the complete list of related objects, you can
// load just a list of their IDs.  To do so, just omit the type declaration when declaring the
// relation:
// <pre>
//    &lt;field name="cities" multiple="true" foreignKey="city.cityId"/&gt;
// </pre>
// When saving, if a replacement list of IDs is included in the Record, the appropriate
// JPA/Hibernate relationships will be updated.
// <P>
// This is very rarely used, and would typically only be used by client-side code that plans to
// programmatically work with the list of related IDs rather than display them in a UI
// component.
// <p>
// <h3><b>NOTE:</b> Bidirectional relations</h3>
// <p>
// When relations are declared, JPA and Hibernate consider only one of the two entities to be
// the "owner" of the relation, meaning essentially that the references that make up the
// relationship are stored with that entity.  When performing updates, make sure you update the
// entity that "owns" the relation.  All changes to relations on "non-owning" entities are
// silently discarded.
//
// <p>
// <h3><b>NOTE:</b> Special case applying IN_SET criteria to One-to-Many and Many-to-Many relations</h3>
// <p>
// For example, Country bean that has a Collection of City beans:
// <pre>
//     &lt;field name="cities" type="city" multiple="true" foreignKey="city.cityId"/&gt;
// </pre>
// To filter it by city.cityId List of IDs must be provided in criteria:
// <pre>
//     criteria:{criteria:[{fieldName:"cities",operator:"inSet",value:[1,2,3]}],operator:"and",_constructor:"AdvancedCriteria"},
// </pre>
// Alternative syntax is using List of Maps where each Map holds idField=value pair:
// <pre>
//     criteria:{criteria:[{fieldName:"cities",operator:"inSet",value:[{cityId:1},{cityId:2},cityId:3]}],operator:"and",_constructor:"AdvancedCriteria"},
// </pre>
// Same approach can be used with Many-to-Many relations, in that case omit foreignKey attribute.
//
// @treeLocation Concepts/Persistence Technologies
// @title JPA &amp; Hibernate Relations
// @visibility external
//<

//> @groupDef manualJpaHibernate
// In some cases you may not be able to immediately use the built-in JPADataSource or
// HibernateDataSource.  For example, you have pre-existing business logic that already
// directly calls JPA or Hibernate APIs, and it may seem to be a large task to refactor this
// logic so that the built-in DataSources can be used.
// <p>
// In this case you can use the overall approach described in
// +link{group:serverDataIntegration} to connect to your pre-existing business logic.  You will
// not be leveraging any of the built-in JPA or Hibernate functionality, so the approach and level of
// effort will be the same as if you were using a non-Hibernate ORM or integrating with
// entirely custom Java classes.
// <P>
// +explorerExample{reusableORMDataSource,This example} shows a Hibernate-based implementation of
// a custom DataSource that implements support for simple Criteria, sorting and data paging,
// but not +link{AdvancedCriteria}, +link{dataSourceField.includeFrom,automatic joins},
// automatic transactions or many other features built into HibernateDataSource.  
// <P>
// Because these features are very valuable and more features are added to the built-in
// DataSources all the time, it's recommended that you refactor your code to use the built-in
// DataSources as soon as you can; refactoring existing business logic as validators, DMIs, or
// custom DataSources is often easier than it looks.  
// <P>
// You can also take the approach of using the built-in DataSources for new entities or
// entities where there is currently no significant business logic, while continuing to use
// your existing code where it's non-trivial to refactor.
//
// @title Manual JPA &amp; Hibernate Integration
// @treeLocation Concepts/Persistence Technologies
// @visibility external
//<


//> @groupDef gaeIntegration
// +externalLink{http://code.google.com/appengine/,GAE} supports
// +externalLink{http://code.google.com/appengine/docs/java/overview.html,Java applications}.
// Google provides a great infrastructure for web applications. It takes care of many web
// application developer's headaches: hardware, operating system support, backups, scaling,
// security, mail etc. To run under GAE your application has to comply to GAE rules.
// The biggest difference is GAE datastore - it is not a relational database - instead they use
// something called +externalLink{http://en.wikipedia.org/wiki/BigTable,BigTable}. To simplify development,
// Google has adopted
// +externalLink{http://www.datanucleus.org/products/accessplatform/,DataNucleus Access Platform}
// to provide
// +externalLink{http://code.google.com/appengine/docs/java/datastore/usingjpa.html,JPA 1.0 support}.
// Because GAE datastore is not a relational database
// +externalLink{http://code.google.com/appengine/docs/java/datastore/usingjpa.html#Unsupported_Features_of_JPA,some JPA features}
// are not supported by this JPA implementation.
// <p/>
// <b>Setting up SmartClient application for GAE</b>
// <p/>
// Under the <code>/WEB-INF</code> directory you have to create a file named
// +externalLink{http://code.google.com/appengine/docs/java/config/appconfig.html,<code>appengine-web.xml</code>}
// which will hold Google's specific settings.<br/>
// One important thing to note: static and dynamic contents will be served from different servers.
// There are
// +externalLink{http://code.google.com/appengine/docs/java/config/appconfig.html#Static_Files_and_Resource_Files, special sections}
// in
// +externalLink{http://code.google.com/appengine/docs/java/config/appconfig.html,<code>appengine-web.xml</code>}
// specifying static and dynamic resources. All resources are duplicated if not specified.
// A single GAE application is limited to 3000 files. A typical SmartClient application consists of many
// resources and it exceeds the limit when duplicated (even with a single theme).
// To run a SmartClient application we have to split resources. Here is an example configuration:<pre>
// &lt;?xml version="1.0" encoding="UTF-8"?&gt;
// &lt;appengine-web-app xmlns="http://appengine.google.com/ns/1.0"&gt;
//     &lt;application&gt;GAE_APPLICATION_NAME&lt;/application&gt;
//     &lt;version&gt;1&lt;/version&gt;
//     &lt;static-files&gt;
//         &lt;include path="/index.jsp"/&gt;
//         &lt;include path="/[MODULE_NAME]/**"/&gt;
//         &lt;exclude path="/[MODULE_NAME]/**.xml"/&gt;
//         &lt;exclude path="/[MODULE_NAME]/**.xsl"/&gt;
//         &lt;exclude path="/[MODULE_NAME]/**.wsdl"/&gt;
//     &lt;/static-files&gt;
//     &lt;resource-files&gt;
//         &lt;include path="/[PATH_TO_DATA_SOURCE_FILES]/**"/&gt;
//         &lt;include path="/[MODULE_NAME]/**.xml"/&gt;
//         &lt;include path="/[MODULE_NAME]/**.xsl"/&gt;
//         &lt;include path="/[MODULE_NAME]/**.wsdl"/&gt;
//     &lt;/resource-files&gt;
// &lt;/appengine-web-app&gt;
// </pre>
// To interact with DataSources an additional servlet mapping has to be added to
// <code>web.xml</code>:<pre>
// &lt;servlet-mapping&gt;
//     &lt;servlet-name&gt;IDACall&lt;/servlet-name&gt;
//     &lt;url-pattern&gt;/[MODULE_NAME]/sc/IDACall&lt;/url-pattern&gt;
// &lt;/servlet-mapping&gt;
// </pre>
// <p/>
// <b>Setting up DataSources</b>
// <p/>
// GAE supports only four types as primary keys:<ul>
// <li><code>java.lang.Long</code></li>
// <li><code>java.lang.String</code></li>
// <li><code>java.lang.String</code> with additional annotations</li>
// <li><code>com.google.appengine.api.datastore.Key</code> <b>not supported by SmartClient</b></li>
// </ul>
// Primary key can not be altered after entity is saved.<br/>
// Entities with primary keys <code>Long</code> or <code>String</code> can not participate in
// transactions and can not be used in relations.
// Here is an example how to declare primary key of type <code>String</code> with additional
// annotations:<pre>
// import java.io.Serializable;
// import javax.persistence.Entity;
// import javax.persistence.GeneratedValue;
// import javax.persistence.GenerationType;
// import javax.persistence.Id;
// import org.datanucleus.jpa.annotations.Extension;
//
// &#64;Entity
// public class Bar
//     implements Serializable
// {
//    &#64;Id
//    &#64;GeneratedValue (strategy = GenerationType.IDENTITY)
//    &#64;Extension (vendorName = "datanucleus", key = "gae.encoded-pk", value = "true")
//    private String id;
// }
// </pre>
// DataSource creation is similar to +link{group:jpaIntegration,standard JPA} with one
// difference: property <b><code>+link{attr:dataSource.serverConstructor,serverConstructor}</code></b>
// should be set to <b><code>com.isomorphic.jpa.GAEJPADataSource</code></b>.<br/>
// Because of
// +externalLink{http://code.google.com/intl/en/appengine/docs/java/datastore/queriesandindexes.html#Restrictions_on_Queries, GAE queries limitations}
// this DataSource implementation supports only single inequality criteria in filter.<br/>
// Only <code>TextMatchStyle.STARTS_WITH</code> filtering mode is supported for text fields.<br/>
// All queries are case sensitive because GAE does not support <code>upper()/lower()</code> functions in criterias.<br/>
// <code>TextMatchStyle.EXACT</code> is used for all number fields.<br/>
// <b><code>com.isomorphic.jpa.EMFProviderLMT</code></b> or
// <b><code>com.isomorphic.jpa.EMFProviderNoTransactions</code></b> should be used as
// transaction providers (depending whether you use transactions or not).<br/>
// To participate in a transaction, entities have
// to belong to the
// +externalLink{http://code.google.com/intl/en/appengine/docs/java/datastore/transactions.html,same group}.<br/>
// Note: entities of different type can not participate in a transaction even if these
// entities have GAE specific primary key (you can not even fetch (SELECT) entities belonging
// to different groups).
// <p/>
// <b>+externalLink{http://code.google.com/intl/en/appengine/docs/java/datastore/relationships.html,Relationships}</b>
// <p/>
// Entities are grouped by establishing owned relationships (where dependent entities are
// instantiated automatically by the JPA provider) between them. Entities in groups can form a
// chain of the following sort::<pre>
// ClassA has reference to ClassB,
// ClassB has reference to ClassC
// </pre>
// But it is impossible to have an entity referencing two other entities:<pre>
// ClassD has reference to ClassE,
// ClassD has reference to ClassF
// </pre>
// There are no foreign keys - the actual reference is encoded into the primary key of child entity.<br/>
// GAE datastore does not support many-to-many relationships.<br/>
// Unidirectional one-to-many relationships work only if the parent has a declaration of
// <code>List&lt;ChildEntityClass&gt;</code>.<br/>
// Unidirectional relationships do not work if only the child entity has reference to the parent.<br/>
// Bidirectional relationship example:<pre>
// &#64;Entity
// public class Country
//     implements Serializable
// {
//     &#64;Id
//     &#64;Column (nullable = false)
//     &#64;GeneratedValue (strategy = GenerationType.IDENTITY)
//     &#64;Extension (vendorName = "datanucleus", key = "gae.encoded-pk", value = "true")
//     private String countryId;
//
//     &#64;OneToMany
//     private List<City> cities;
// //....
// }
//
// &#64;Entity
// public class City
//     implements Serializable
// {
//     &#64;Id
//     &#64;Column (nullable = false)
//     &#64;GeneratedValue (strategy = GenerationType.IDENTITY)
//     &#64;Extension (vendorName = "datanucleus", key = "gae.encoded-pk", value = "true")
//     private String cityId;
//
//     // This is fake column - it is calculated by provider and is not saved.
//     // Actual reference to parent entity is encoded in primary key.
//     &#64;Column (nullable = false)
//     &#64;Extension (vendorName = "datanucleus", key = "gae.parent-pk", value = "true")
//     private String countryId;
//
//     &#64;ManyToOne (fetch=FetchType.LAZY)
//     private Country country;
// //....
// }
// </pre>
// Note: GAE does not support <code>FetchType.EAGER</code>.
// <p/> With
// <b>+externalLink{http://code.google.com/intl/en/appengine/docs/java/datastore/relationships.html#Unowned_Relationships,Unowned Relationships}</b>
// (when you save parent's primary key as simple child's property) you can model unsupported
// relationships. But this approach has drawbacks:<ul>
// <li>related entities are not instantiated automatically</li>
// <li>transactions can not be used</li>
// <li>you have to manually keep track of changes in case of bidirectional relationship</li>
// </ul>
//
// @see group:jpaIntegration
// @treeLocation Concepts/Persistence Technologies
// @title Google Application Engine (GAE)
// @visibility external
//<

//> @groupDef sqlConnectionPooling
// Many databases will automatically close inactive connections, which can interfere with
// connection pooling: if an application is not constantly using all of the connections in its
// pool, it may retrieve a closed connection from the pool.
// <P>
// In some cases you can disable the behavior of closing inactive connections.  For MySQL it's
// controlled by the
// +externalLink{http://dev.mysql.com/doc/refman/5.0/en/server-system-variables.html#sysvar_wait_timeout,wait_timeout}
// setting in your my.cnf file).  However, this could potentially cause leaked connections if
// applications terminate without cleaning up their database connections.
// <P>
// Intelligent connection pools compensate for unexpectedly closed connections automatically:
// <ul>
// <li>J2EE containers generally implement internal keepalives or staleness checks - this is
// the preferred solution if available.  If using SQLDataSource, use JNDI-based
// configuration as described +link{group:adminConsole,here}.
// <li>SQLDataSource uses DBCP (Apache Commons) pooling, which also compensates for connection
// closure automatically.  This is enabled by default with appropriate settings, but can be
// disabled system wide via setting <b>sql.pool.enabled</b> to false in
// +link{server_properties,server.properties}, or
// disabled for a specific database configuration via <b>sql.<i>dbName</i>.pool.enabled</b>.
// The following properties can also be set on sql.pool / sql.<i>dbName</i>.pool and control
// same-named DBCP properties, however, it is not recommended to set these properties unless
// you have experience with DBCP and are troubleshooting a specific pool-related performance
// problem: testOnBorrow, testOnReturn, testWhileIdle, timeBetweenEvictionRunsMillis,
// minEvictableIdleTimeMillis, numTestsPerEvictionRun.
// <li>JPA/Hibernate: Hibernate's built-in connection pool is <b>not</b> intended for
// production use according to Hibernate's own documentation.  This includes using JPA with
// Hibernate as the provider. If you get dead connections during development you can disable
// Hibernate's built-in connection pool by setting "hibernate.connection.pool_size" to 0. For
// production use you must use production-ready connection pool libraries for example
// C3P0. Here are recommended settings for C3P0 properties:
// <ul>
// <li>c3p0.acquireRetryDelay=1000</li>
// <li>c3p0.acquireRetryAttempts=60</li>
// <li>c3p0.breakAfterAcquireFailure=false</li>
// </ul>
// </li>
// </ul>
// @treeLocation Concepts/Persistence Technologies
// @title SQL Connection Pooling
// @visibility external
//<

//> @groupDef standaloneDataSourceUsage
// The DataSource layer can be used in a standalone application separate from a servlet engine,
// for things like Java Swing applications, batch data processing or sending emails based on
// database state.
// <p>
// Note that we do still require <code>servlet-api.jar</code> to be on the classpath of 
// standalone applications.  This is only to satisfy class-loading requirements; there are 
// cases where running inside a servlet engine allows us to provide extra functionality, such
// as declarative authentication and authorization features, so some classes do have member 
// variables or method parameters of types in the Servlet API.  They all operate correctly 
// when these variables or parameters are null, but the JVM will still try to load the classes 
// and will crash if they are missing.
// <p> 
// Using the DataSource layer to run DataSource operations in your standalone applications is
// extremely straightforward.  This example fetches and prints every record from the 
// "customers" DataSource:
// <pre>
//   public static void main(String[] args) {
//     DataSource ds = DataSourceManager.get("customers");
//     List records = ds.fetch();
//     for (Iterator i = records.iterator; i.hasNext(); ) {
//       System.out.println(i.next());
//     }
//   }
// </pre>
// To make this example fetch just customers in the United States:
// <pre>
//     Map criteria = new HashMap();
//     criteria.put("countryCode", "US");
//     List records = ds.fetch(criteria);
// </pre>
// This example shows how to run a specific fetch operation, specifying both selection 
// criteria and a sort order, using a <code>DSRequest</code> rather than a <code>DataSource</code> 
// convenience method:
// <pre>
//   public static void main(String[] args) {
//     DSRequest dsReq = new DSRequest("customers", "fetch");
//     dsReq.setOperationId("specialFetch");
//     Map criteria = new HashMap();
//     criteria.put("countryCode", "US");
//     criteria.put("region", "CA");
//     dsReq.setCriteria(criteria);
//     dsReq.setSortBy("customerName");
//     List records = dsReq.execute().getDataList();
//   }
// </pre>
// This example shows how to do a simple update:
// <pre>
//   public static void main(String[] args) {
//     DataSource ds = DataSourceManager.get("customers");
//     Map criteria = new HashMap();
//     criteria.put("customerNo", 12345);
//     Map values = new HashMap();
//     values.put("creditLimit", 10000);
//     values.put("currencyCode", "USD");
//     ds.update(criteria, values);
//   }
// </pre>
// Finally, this example shows how to perform a specific update operation via a 
// <code>DSRequest</code>:
// <pre>
//   public static void main(String[] args) {
//     DSRequest dsReq = new DSRequest("customers", "update");
//     dsReq.setOperationId("specialUpdate");
//     Map criteria = new HashMap();
//     criteria.put("customerNo", 12345);
//     Map values = new HashMap();
//     values.put("creditLimit", 10000);
//     values.put("currencyCode", "USD");
//     dsReq.setCriteria(criteria);
//     dsReq.setValues(values);
//     dsReq.execute();
//   }
// </pre>
// <b>NOTES</b>
// <p>
// Because we are not running inside a servlet container, SmartClient's built-in logic to 
// work out where its application root is does not work.  Therefore, you need to manually 
// set a "webRoot" in your +link{server_properties,server.properties} file. 
// The webRoot should point
// to the root folder of your application (note for SmartGWT applications, this is typically
// the "war" subfolder of your project).  Example entries:<p>
// <code>    webRoot: /home/testUser/myProject</code><p>
// or:<p>
// <code>    webRoot: C:\Projects\MyProject\war</code><p>
// Again in +link{server_properties,server.properties}, 
// you may need to set <code>isomorphicPathRootRelative</code>
// to match the standalone project's layout if you make the standalone mode separate:<p>
// <code>    isomorphicPathRootRelative: myProject/sc</code>
// <p>
// <var class="smartgwt">
// Note that the directory structure that normally appears in "war/<i>myProject</i>/sc" in your
// GWT project is required to be present under the configured
// <i>webRoot</i>+<i>isomorphicPathRootRelative</i> directory when running standalone.  This structure is
// normally created by the GWT compiler, by processing &lt;inherits&gt; tags in your .gwt.xml
// that cause resources to be copied from smartgwtee.jar.  If your build/deployment scripts do
// not invoke the GWT compiler these files will not be present, so you will need to either add
// a step to invoke the GWT compiler or manually copy the files.
// </var>
// You should place the +link{server_properties,server.properties} file somewhere on your classpath.  
// Typically, the root of your <code>bin</code> or <code>classes</code> folder structure is 
// the most suitable place for it.
// <p>
// Both the built-in DataSources and custom DataSources can be used in standalone
// applications, <b>but only if you do not introduce dependencies on the servlet container in
// your DataSource code</b>. For example, if you have a security check in a DMI or custom
// DataSource that depends on checking the current logged-in user, code in a means of bypassing
// this, or create a parallel operationBinding that is accessible only to the superuser.
// <p>
// When you use the DataSource layer in a standalone application, 
// +link{dataSource.requiresAuthentication,Declarative Security} is disabled; all operations
// are considered to be valid for use in a standalone application.  Other DataSource features
// that are disabled because they are not relevant in a standalone context are:
// <ul>
// <li>+link{DSRequestModifier.value,Transaction Chaining}</li>
// <li>+link{DataSource.autoJoinTransactions,Transactional persistence}</li>
// </ul>
// <br>
// <b>Using Spring in a standalone application</b>
// <p>
// In a typical web application, Spring configuration is picked up from an "applicationContext" 
// file by a servlet or listener, and then made available to the rest of the app via the 
// servletContext.  When running standalone, this is not possible, so instead we read the 
// applicationContext file manually when we need to, eg, create a DataSource object that is 
// configured as a Spring bean.
// <p>
// By default, the framework will look in the "normal" place for for this configuration: 
// <code>WEB-INF/applicationContext.xml</code>.  If you have changed the location or name 
// of this file, and you want to run the application outside of a servlet engine, you can tell
// the framework where to find the configuration file by specifying property 
// <code>standalone.spring.applicationContext</code> in your +link{server_properties,server.properties}.
// The default setting for this property looks like this:<pre>
//    standalone.spring.applicationContext: $webRoot/WEB-INF/applicationContext.xml
// </pre>
//
// @treeLocation Client Reference/Data Binding/DataSource
// @title Standalone DataSource Usage
// @visibility external
//<



//> @groupDef sqlDataSource
// The SmartClient Server supports comprehensive, codeless SQL connectivity for DataSources
// defined in XML.  Our SQL connectivity is mature, feature-rich, protected against injection
// attacks and easily customizable to support user-written SQL and additional business logic
// of any complexity.  +link{group:sqlVsJPA,This article} compares the built-in SQL DataSource
// to other persistence approaches based on Javabeans.
// <P>
// To use the built-in SQL engine, declare a +link{DataSource} in XML format with
// +link{dataSource.serverType} set to "sql", and place it in the shared dataSources directory
// ([webroot]/shared/ds by default) in a file called "[dataSourceId].ds.xml".
// <p>
// You can then use the +link{group:adminConsole,Admin Console} to configure database access,
// as well as to automatically create and populate a database table based on your DataSource.
// By default, DataSources will use the "default database" from the admin console, however you
// can set +link{dataSource.dbName} to the name of a specific database configuration you have
// configured via the Admin Console.
// <P>
// The list of databases known to work with the the built-in SQL logic is as follows:
// <p>
// <table class='normal'>
// <tr><td width=40></td><td width=400><i>HSQLDB 1.7.x, 1.8.x, 2.0.x, 2.2.x, 2.3.x</i></td><td></td></tr>
// <tr><td></td><td><i>IBM DB2 8.x, 9.x, 10.x</i></td><td></td></tr>
// <tr><td></td><td><i>IBM DB2 for i (formerly known as DB2 for i5/OS) V5R4M0 onwards</i></td><td></td></tr>
// <tr><td></td><td><i>Firebird 2.5</i></td><td></td></tr>
// <tr><td></td><td><i>Informix 11.5+, 12.1</i></td><td></td></tr>
// <tr><td></td><td><i>MS SQL Server 2000, 2005, 2008, 2008 R2, 2012, 2014</i></td><td></td></tr>
// <tr><td></td><td><i>MySQL 3.2.x, 4.0.x, 4.1.x, 5.0.x, 5.1.x, 5.5.x, 5.6x</i></td><td></td></tr>
// <tr><td></td><td><i>Oracle 8.0.5, 8i, 9i, 10g, 11g, 12c</i></td><td></td></tr>
// <tr><td></td><td><i>PostgreSQL 7.x, 8.x, 9.x</i></td><td></td></tr>
// </table>
// <p>We also support a generic SQL92 database connection which works for basic CRUD operations
// with any database product that supports standard SQL92 syntax and data types, plus a couple 
// of widely-implemented features that are not actually part of the standard.  Specifically,
// this means we do not support:<ul>
// <li>Sequences</li>
// <li>Paging via SQL limit queries</li>
// <li>+link{DataSource.autoJoinTransactions,Automatic transaction management}
// <li>Long text values (there is no real definition of "long" here - we try to use a standard
// VARCHAR, but different databases will support different maximum values for this)</li>
// <li>Databases that do not implement the widely-supported LOWER() function</li>
// <li>Databases that do not support the ability to perform string-type operations on numeric
// columns - for example, <code>myNumericColumn LIKE '%5%'</code></li>
// </ul>
// <p>You will also need a JDBC driver for your specific database. Licensing restrictions
// prevent us including any JDBC driver other than the one for HSQLDB.  However, you
// can download these drivers for free from the vendors' websites.  If your specific database
// server or version is not listed above, please go to the
// <a href="http://forums.smartclient.com">SmartClient forums</a> for assistance.
// <P>
// You can also connect to an existing database table - the Visual Builder DataSource Wizard
// ("New" button in DataSources pane) allows you to pick an existing database table and have a
// DataSource generated from it.  The Batch DataSource Generator tool will allow you to
// generate DataSources for any number of SQL DataSources, and customize the process so that
// organization-specific rules are applied (eg ignoring certain columns, or using naming
// conventions to generate more complete DataSources).
// <P>
// Once you have your SQL DataSource connected to a table, in a default SDK installation,
// DSRequests for your DataSource will be sent to the default
// +link{RPCManager.actionURL,actionURL}, and hence handled automatically, without you having
// to write any Java code, by the +link{servletDetails,IDACall servlet registered in web.xml}.
// IDACall is a very simple servlet that just calls the server-side method
// dsRequest.execute() for all inbound requests.  For more details on how DataSource requests
// are processed by SmartClient Server, and how you can alter and add to this processing, see
// this description of +link{group:serverDataIntegration,server data integration}.
//
// @treeLocation Client Reference/Data Binding/DataSource
// @title SQL DataSources
// @requiresModules SCServer
// @visibility external
//<

//> @groupDef adminConsole
// The Admin Console allows you to configure database access for DataSources that use
// SmartClient's built-in +link{group:sqlDataSource,SQL engine}.  You can either use the Admin
// Console UI (as explained below) or directly specify equivalent properties in your
// +link{server_properties,server.properties} file (see "Manually specifying.." below).
// <P>
// <var class="smartclient">
// NOTE: To use the Admin Console, you must have the Isomorphic SmartClient package installed
// and your servlet engine started. 
// <p>
// Direct your browser to the following URL to access the Admin Console: 
// <p>
// &nbsp;&nbsp;+externalLink{http://localhost:8080/tools/adminConsole.jsp}
// <p>
// The common default servlet engine port 8080 is used in the URL given above. Adjust your URL
// as necessary if you are using a different port and replace localhost with the machine name
// running the servlet engine if you are accessing it from a remote machine.
// </var>
// <var class="smartgwt">
// To use the Admin Console, make sure you have followed the 
// +link{group:sgwtEESetup,full installation instructions} - double check that you have the
// correct GWT &lt;inherits&gt; as shown in those instructions.  Then, launch your GWT project
// from your IDE, and open a web browser to <i>moduleBaseURL</i>/tools/adminConsole.jsp.  For
// example, if you load your application by going to a URL that ends in
// builtinds/builtinds.html, the correct URL for the Admin Console is builtinds/tools/adminConsole.jsp.
// <P>
// If it's not clear what URL to use, you can add the following code to your onModuleLoad()
// method to create a button that opens the Admin Console:
// <pre>
//   IButton adminButton = new IButton("Admin Console"); 
//   adminButton.addClickHandler(new ClickHandler() { 
//     public void onClick(ClickEvent event) { 
//       com.smartgwtee.tools.client.SCEE.openDataSourceConsole(); 
//     } 
//   }); 
//   adminButton.draw();
// </pre>
// NOTE: if you are using Pro Edition, the method to call is
// com.smartgwtpro.tools.client.SCPRO.openDataSourceConsole(), and for Power Edition,
// com.smartgwtpower.tools.client.SCPower.openDataSourceConsole().
// </var>
// Having opened the Admin Console, available JNDI connections will be discovered and shown
// automatically.  If you aren't using JDNI, use the GUI to enter and test JDBC settings. Both
// ConnectionManager and JDBC DataSource settings are supported. Once you've got a working
// connection, set it as the default connection using the "Set as Default" button.
// <var class="smartgwt">
// <P>
// These settings will be written to +link{server_properties,server.properties}
// in your deployment directory - use
// the "Download server.properties" button to download the settings and merge them to the
// server.properties file in your Eclipse (or other IDE) project.
// </var>
// <P>
// <b>Test Data</b>
// <p>
// You can create a test file that contains a sample dataset which can be imported into your
// database table with the Admin Console.
// <p>
// The test file to use with your DataSource is specified in the <code>testFileName</code>
// DataSource configuration property. The test file uses the extension .data.xml.
// <p>
// The test data file should consist of a top-level &lt;List&gt; element containing a series of XML
// elements named after your DataSource's ID, each of which creates one DataSource
// record. Values for each field are given within tags named after the field name. 
// <p>
// For example, the following XML is from the supplyItem.data.xml test data file supplied with
// the Isomorphic SmartClient package. This file is located in
// [webroot]/examples/shared/ds/test_data/.
// <p>
// <pre>
// &lt;List&gt;
//  &lt;supplyItem&gt;
//      &lt;description&gt;A revolutionary cushion-grip ballpoint pen that reduces 
//          required gripping power, relieving stress and alleviating writing 
//          fatigue. Excellent for people who suffer from arthritis or carpal 
//          tunnel syndrome. Medium point, black ink. Refillable.&lt;/description&gt;
//      &lt;category&gt;1&lt;/category&gt;
//      &lt;itemRef&gt;ODC 204-502-153&lt;/itemRef&gt;
//      &lt;maxQuantity&gt;5&lt;/maxQuantity&gt;
//      &lt;requiresJustification&gt;0&lt;/requiresJustification&gt;
//      &lt;itemName&gt;Dr. Grip Pens -- Blue Barrel&lt;/itemName&gt;
//      &lt;itemID&gt;1&lt;/itemID&gt;
//      &lt;unitCost&gt;4.99&lt;/unitCost&gt;
//  &lt;/supplyItem&gt;
//  &lt;supplyItem&gt;
//      &lt;description&gt;A revolutionary cushion-grip ballpoint pen that reduces 
//          required gripping power, relieving stress and alleviating writing 
//          fatigue. Excellent for people who suffer from arthritis or carpal 
//          tunnel syndrome. Medium point, black ink. Refillable.&lt;/description&gt;
//      &lt;category&gt;1&lt;/category&gt;
//      &lt;itemRef&gt;ODC 204-708-834&lt;/itemRef&gt;
//      &lt;maxQuantity&gt;5&lt;/maxQuantity&gt;
//      &lt;requiresJustification&gt;0&lt;/requiresJustification&gt;
//      &lt;itemName&gt;Dr. Grip Pens -- Black Barrel&lt;/itemName&gt;
//      &lt;itemID&gt;2&lt;/itemID&gt;
//      &lt;unitCost&gt;4.99&lt;/unitCost&gt;
//  &lt;/supplyItem&gt;
//  &lt;supplyItem&gt;
//      &lt;description&gt;Personalized business cards for all your networking 
//          needs.&lt;/description&gt;
//      &lt;category&gt;2&lt;/category&gt;
//      &lt;itemRef&gt;&lt;/itemRef&gt;
//      &lt;maxQuantity&gt;500&lt;/maxQuantity&gt;
//      &lt;requiresJustification&gt;1&lt;/requiresJustification&gt;
//      &lt;itemName&gt;Personalized business cards -- 500 count&lt;/itemName&gt;
//      &lt;itemID&gt;3&lt;/itemID&gt;
//      &lt;unitCost&gt;25.00&lt;/unitCost&gt;
//  &lt;/supplyItem&gt;
//  ...
// &lt;List/&gt;
// </pre>
// 
// Data for a tree-like DataSource can be specified with the same format.
// The following code example is from the supplyCategory.data.xml test data file. This file
// is also located in [webroot]/examples/shares/ds/test_data/. 
//
// <pre>
// &lt;List&gt;
//     &lt;supplyCategory&gt;
//     	&lt;itemName&gt;Office Paper Products&lt;/itemName&gt;
//     	&lt;parentID&gt;root&lt;/parentID&gt;
//     &lt;/supplyCategory&gt;
//     &lt;supplyCategory&gt;
//     	&lt;itemName&gt;Calculator Rolls&lt;/itemName&gt;
//     	&lt;parentID&gt;Office Paper Products&lt;/parentID&gt;
//     &lt;/supplyCategory&gt;
//     &lt;supplyCategory&gt;
//     	&lt;itemName&gt;Adding Machine/calculator Roll&lt;/itemName&gt;
//     	&lt;parentID&gt;Calculator Rolls&lt;/parentID&gt;
//     &lt;/supplyCategory&gt;
//     . . .
// &lt;/List&gt;
// </pre>
// 
// Notice that all records must define values for the itemName primary key field and for the
// parentID field that establishes the tree relationship.
// <P>
// <br>
// <b>Manually specifying database connection settings</b>
// <p>
// The Admin Console maintains settings in the +link{server_properties,server.properties}
// file, found in 
// your application's <code>WEB-INF/classes</code> directory.  If you prefer, you can maintain
// these settings by directly editing that file.  You should restart your servlet engine 
// after changing this file.
// <p>
// For example, the following settings are the defaults in a new SmartClient installation for 
// a MySQL server; they are approximately correct for a MySQL server running on the same 
// machine as the servlet engine and listening on the default MySQL port.  For details of what
// each of these properties means, check +link{group:sqlSettings,this page}.<pre>
//   sql.Mysql.database.type: mysql
//   sql.Mysql.database.ansiMode: false
//   sql.Mysql.interface.type: dataSource
//   sql.Mysql.driver: com.mysql.jdbc.jdbc2.optional.MysqlDataSource
//   # name of the database to use
//   sql.Mysql.driver.databaseName: isomorphic
//   # hostname and port where the database server is installed
//   sql.Mysql.driver.serverName: localhost
//   sql.Mysql.driver.portNumber: 3306
//   # username and password that can create and modify tables in that database
//   # this user must have the following privileges for the system to function
//   # properly: create/alter/drop table; insert/update/replace/delete rows.
//   sql.Mysql.driver.user: root
//   sql.Mysql.driver.password: 
// </pre>
// Note the distinction here between database <em>type</em> and database <em>name</em>.  Database
// type refers to the actual product - Oracle, DB2 or whatever.  In the above example, database 
// type is "mysql" (all lowercase) - the value of property <code>sql.Mysql.database.type</code>.
// Database type is very important.  The type of a given database connection dictates whether
// features like SQL paging and transactions are supported; it even dictates the syntax of the 
// SQL we generate.
// <p>
// Database name is just an arbitrary name for a particular database connection, and it is
// embedded in the property names immediately after the <code>sql</code> prefix. In this example
// it happens to be very similar to the database type - "Mysql" as opposed to "mysql" - but in 
// fact the name has no significance and could be any string.  When referring to specific 
// database connections in your +link{class:DataSource,DataSources} with the 
// +link{DataSource.dbName,dbName} property, it is the database <em>name</em> you use.
// <p>
// NOTE: It is common for DataSources to not specify <code>dbName</code>.  In this case, the 
// default database is used.  To specify the default database manually in 
// +link{server_properties,server.properties}, set <code>sql.defaultDatabase</code>, using database 
// name.  So, to set our example connection from above as the default:<pre>
//   sql.defaultDatabase: Mysql
// </pre>
// <P>
// <b>Manually specifying JNDI settings</b>
// <p>
// Instead of specifying database connection parameters directly in 
// +link{server_properties,server.properties},
// it is possible to connect to a database that is configured as a JNDI resource in your 
// application server.  Assume you have an Oracle JNDI resource with the name "jndiTest", 
// configured similar to this in Tomcat:
// <pre>
//   &lt;Resource name="jdbc/jndiTest"
//                    auth="Container"
//                    type="javax.sql.DataSource"
//                    driverClassName="oracle.jdbc.driver.OracleDriver"
//                    url="jdbc:oracle:thin:@192.168.132.152:1521:xe"
//                    username="system"
//                    password="manager"
//                    initialSize="5"                 
//                    maxActive="50" /&gt;
// </pre>
// The minimal set of properties required to create a SmartClient database connection that 
// attaches to this resource is as follows (Note that the <code>java:comp/env/</code> prelude
// in the first line is optional - the server will automatically look there if it can't find 
// the resource in the absolute location)
// <pre>
//   sql.myOracleConnection.driver.name: java:comp/env/jdbc/jndiTest
//   sql.myOracleConnection.database.type: oracle
//   sql.myOracleConnection.interface.type: jndi
// </pre>
//
//
// @see group:sqlConnectionPooling
// @treeLocation Client Reference/Data Binding/DataSource
// @title Admin Console
// @requiresModules SCServer
// @visibility external
//<

//> @groupDef sqlSettings
// Although the Admin Console provides a UI to let you to configure database access for 
// DataSources that use SmartClient's built-in +link{group:sqlDataSource,SQL engine}, it is 
// also possible to configure these DataSources with manual entries in your 
// +link{server_properties,server.properties} file.
// <p>
// When you manually configure a DataSource like this, you do so by maintaining a set of 
// properties with names structured like this:
// <pre>   sql.{dbName}.x.y
// </pre>
// where <code>{dbName}</code> is the name of the database configuration you are providing.
// Note that this database name is just an arbitrary name for a particular database 
// configuration; many of the default ones provided with SmartClient are named after a database
// <em>type</em>, in order to make their intended use more immediately obvious, but this is 
// not by any means a requirement.
// <p>
// For the remainder of this discussion, we will assume we are configuring a database with a 
// name of "MyDatabase".
// <p>
// <h4>SQL configuration properties</h4><p>
// <b><code>sql.MyDatabase.database.type</code></b><br>
// This should be set to one of the supported database types.  These are:<p>
// <table style="font-size:10px">
// <tr><td><b>hsqldb</b></td><td>HSQLDB 1.7.x and greater</td></tr>
// <tr><td><b>db2</b></td><td>IBM DB2 8.x and greater</td></tr>
// <tr><td><b>db2iSeries</b></td><td>IBM DB2 for iSeries/i5, V5R4 and greater</td></tr>
// <tr><td><b>firebirdsql</b></td><td>Firebird 2.5 and greater</td></tr>
// <tr><td><b>informix</b></td><td>Informix 11.5 and greater</td></tr>
// <tr><td><b>sqlserver</b></td><td>Microsoft SQL Server 2000 and greater</td></tr>
// <tr><td><b>mysql</b></td><td>MySQL 3.2.x and greater</td></tr>
// <tr><td><b>oracle</b></td><td>Oracle 8.0.5, 8i and greater</td></tr>
// <tr><td><b>postgresql</b></td><td>PostgreSQL 7.x and greater</td></tr>
// <tr><td><b>generic</b></td><td>A generic SQL92 database, with limitations described in
// +link{group:sqlDataSource,this article}</td></tr>
// </table><p>
// <code><b>sql.MyDatabase.driver</b></code><br>
// The name of the JDBC driver implementation.  This depends upon your database product and 
// version, and the specific JDBC driver you are using (JDBC drivers can usually be downloaded
// from your database vendor's website).  Bearing in mind the caveat that this information can
// vary by release and JDBC implementation, here are some suggested values for our supported 
// databases:<p>
// <table style="font-size:10px">
// <tr><td><b>hsqldb</b></td><td><code>org.hsqldb.jdbcDriver</code></td></tr>
// <tr><td><b>db2</b></td><td><code>com.ibm.db2.jcc.DB2DataSource</code></td></tr>
// <tr><td><b>db2iSeries</b></td><td><code>com.ibm.as400.access.AS400JDBCDriver</code></td></tr>
// <tr><td><b>firebirdsql</b></td><td><code>org.firebirdsql.jdbc.FBDriver</code></td></tr>
// <tr><td><b>informix</b></td><td><code>com.informix.jdbc.IfxDriver</code></td></tr>
// <tr><td><b>sqlserver</b></td><td><code>com.microsoft.jdbc.sqlserver.SQLServerDriver</code> or 
// <code>com.microsoft.sqlserver.jdbc.SQLServerDriver</code> (Microsoft changed the order of 
// "jdbc" and "sqlserver" between the 2000 and 2005 editions of the product)</td></tr>
// <tr><td><b>mysql</b></td><td><code>com.mysql.jdbc.jdbc2.optional.MysqlDataSource</code></td></tr>
// <tr><td><b>oracle</b></td><td><code>oracle.jdbc.driver.OracleDriver</code></td></tr>
// <tr><td><b>postgresql</b></td><td><code>org.postgresql.Driver</code></td></tr>
// </table><p>
// <code><b>sql.MyDatabase.driver.serverName</b></code><br>
// The name or IP address of the database server
// <p>
// <code><b>sql.MyDatabase.driver.portNumber</b></code><br>
// The port on which the database server is listening
// <p>
// <code><b>sql.MyDatabase.driver.user</b></code><br>
// The user to connect as
// <p>
// <code><b>sql.MyDatabase.driver.password</b></code><br>
// The user's password
// <p>
// <code><b>sql.MyDatabase.driver.databaseName</b></code><br>
// The database to connect to.  A "database" in this context is a named collection of tables
// and other database resources that are somehow grouped together by the database product.
// The specifics of how this is implemented vary by database.  Note that some database 
// products use the terms "catalog" or "schema" to refer to the same concept, and Oracle - 
// although it does also have a concept of catalog - uses the term "SID" for this concept.<p>
// <code><b>sql.MyDatabase.interface.type</b></code><br>
// Indicates how the JDBC connection will be created or looked up; the value of this setting 
// depends on the capabilities of the particular JDBC driver you are using, and is inherently
// connected to the value of <code>sql.MyDatabase.driver</code>.  The following settings are 
// supported:<p>
// <b>dataSource</b> - the driver is an instance of <code>javax.sql.DataSource</code> and 
// should be instantiated by SmartClient Server<br>
// <b>driverManager</b> - the driver is an instance of <code>java.sql.DriverManager</code><br>
// <b>jndi</b> - the driver is an instance of <code>javax.sql.DataSource</code> and should be 
// looked up using JNDI<p>
// <code><b>sql.MyDatabase.driver.url</b></code><br>
// For configurations where <code>sql.MyDatabase.interface.type</code> is "driverManager", 
// this property allows you to manually enter the URL we use to connect to the database.  If 
// this property is not provided, we build the URL from other settings such as 
// <code>sql.MyDatabase.driver.serverName</code> and 
// <code>sql.MyDatabase.driver.databaseName</code>.<p>
//
// <b>Other properties</b><br>
// Different JDBC drivers support different properties to support product-specific quirks and
// features.  You can often specify these properties by embedding them as parameters in the 
// URL used to connect to the database.<p>
//
// Alternatively, any subproperty you set on the "driver" in server.properties is applied to 
// the JDBC driver object via Reflection. For example, the MySQL JDBC driver supports a property
// "useUnicode", which forces the database to use Unicode character encoding.  If 
// <code>sql.MyDatabase.driver</code> is <code>com.mysql.jdbc.jdbc2.optional.MysqlDataSource</code>,
// setting <code>sql.MyDatabase.driver.useUnicode</code> to true means we'll attempt to call 
// <code>setUseUnicode(true)</code> on this class.  This would have exactly the same effect as
// defining the connection URL manually and specifying the parameter <code>useUnicode=true</code>
//
// @treeLocation Client Reference/Data Binding/DataSource
// @title SQL Database Settings in <code>server.properties</code>
// @requiresModules SCServer
// @visibility external
//<

//> @groupDef clientOnlyDataSources
// For prototyping purposes, a "client-only" DataSource can be created that has no permanent
// storage and never contacts the server, instead using a set of test data to respond to
// requests in the same manner as a server-based DataSource might.
// <P>
// The client-side interface to a client-only DataSource is identical to a server-based
// DataSource, including asynchronous responses, so that a client-only DataSource can be
// replaced by a server-based DataSource without code changes.  The only difference is that
// changes to records belonging to the DataSource persist only until the page is reloaded.
// <p>
// Client-only DataSources allow you to create a complete prototype application in an .html
// file that does not require a server.
// <p>
// <var class="smartclient">
// The <code>clientOnly</code> property is specified to create a client-only DataSource, and
// the <code>testData</code> property should contain the test dataset, as an Array of Objects,
// one per DataSource record.  For example:
// <pre>
//   isc.DataSource.create({
//       ID:"supplyItem",
//       fields: ...,
//       clientOnly:true,
//       testData:[
//          {itemName:"Pencil", cost:5.50},
//          ...
//       ]
//   });
// </pre>
// If you have existing test data in XML (see the +link{group:adminConsole,Admin Console} for
// expected format),
// you can use the XML->JS translation engine to load it into a client-only DataSource, like so:
// <pre>
//   isc.DataSource.create({
//     ID:"solutions",
//     fields: ...,
//     clientOnly : true,
//     testData : 
//         &lt;isomorphic:XML filename="shared/ds/test_data/solutions.data.xml"/&gt;
//   });
// </pre>
// Another useful practice is to specify both the clientOnly DataSource and it's test data in
// XML, so that the +link{group:adminConsole,Admin Console} can later be used to import the
// DataSource and its test data into a SQL Database.  An idiom for accomplishing this is:
// <pre>
//   &lt;isomorphic:loadDS name="solutions"/&gt;
//   isc.DataSource.getDataSource("solutions").addProperties({
//     clientOnly : true,
//     testData : 
//        &lt;isomorphic:XML filename="shared/ds/test_data/solutions.data.xml"/&gt;
//   });
// </pre>
// If you specify your DataSource as <code>clientOnly: true</code>, omit testData
// entirely, and provide either a +link{attr:dataSource.dataURL} or a <code>testFileName</code>, the
// </var>
// <var class="smartgwt">
// The setClientOnly method is specified to create a client-only DataSource. For example:
// <pre>
//  DataSource dataSource = new DataSource();
//  dataSource.setID("stockQuotesDS");
//  DataSourceField idField = new DataSourceField("id", FieldType.INTEGER, "Id");
//  ........
//  dataSource.setFields(idField,.....);
//  dataSource.setClientOnly(true);
// </pre>
// If you have existing test data in XML (see the Admin Console for expected format), 
// you can load it into a client-only DataSource using setDataURL method, like so:
// <pre>
//  DataSource dataSource = new DataSource();
//  dataSource.setID("stockQuotesDS");
//  dataSource.setRecordXPath("/List/stockQuotes");
//  DataSourceField idField = new DataSourceField("id", FieldType.INTEGER, "Id");
//  ........
//  dataSource.setFields(idField,.....);
//  dataSource.setDataURL("ds/test_data/stockQuotes.data.xml");
//  dataSource.setClientOnly(true);
// </pre>
// The
// </var>
// DataSource will lazily make a one-time fetch against the specified data file the first time
// an operation is called on it.  From then on, the DataSource will work against the local
// cache created from this initial request.  This is a quick way to prototype against some test
// data that may eventually be returned from an arbitrary back-end.
// <P>
// Finally, it is possible to have a DataSource which initially fetches the entire dataset and
// performs all subsequent fetching locally, while still visiting the server to perform all 
// other operations.  See +link{dataSource.cacheAllData}.
//
// @treeLocation Client Reference/Data Binding/DataSource
// @title Client Only DataSources
// @visibility external
//<


// synonym, isc namespace only (never a global)
isc.DS = isc.DataSource;

// static properties and methods
isc.DataSource.addClassProperties({
    

	// locating dataSources
	dataSourceObjectSuffix : "DS",			// suffix for the global to hold info about a dataSource,
											//	eg:  var myDataSourceDS = {...}
	
	_dataSources : {},						// internal array to hold the list of dataSources we've loaded
	_elements : {},						    // internal array of XSElements
	_types : {},						    // internal array of XSComplexTypes
    _$element : "element", _$type:"type",
    _tableCodesToDatasourceIds: {},         // a map to find loaded sql datasource by it's tableCode
    _fkTableCodesToDatasourceIds: {},
	
	TABLE : "table",						// table type of datasource
	VIEW : "view",							// view type of datasource (virtual datasource)

    //> @classAttr DataSource.loaderURL (URL : DataSource.loaderURL : RW)
    //
    // The URL where the DataSourceLoader servlet has been installed.  Defaults to the
    // +link{Page.setIsomorphicDir,isomorphicDir} plus "/DataSourceLoader".  Change via
    // addClassProperties:
    // <pre>
    //    isc.DataSource.addClassProperties({ loaderURL: "newURL" });
    // </pre>
    //
    // @visibility external
    //<
	loaderURL:"[ISOMORPHIC]/DataSourceLoader",
	
	// deepCloneOnEdit: If we have nested dataSources, and have edit-components bound to these
	// DS's, using dataPath to extract values from inner objects, should the field values be
	// recursively duplicated on edit.
	// Default value is true - this is required to support standard functionality such as being
	// able to pass a ListGrid record to a DynamicForm for editing and avoiding having both 
	// components referencing the same inner-objects in memory [thus leading to edits directly
	// effecting the ListGrid record with no save-step or other notification].
	
	deepCloneOnEdit:true,

	// See class method registerSgwtDS()
    DS_FN: {},

    // chunks of SOAP messages
	_soapEnvelopeStart : "<soap:Envelope xmlns:soap='http://schemas.xmlsoap.org/soap/envelope/' ",
	_soapEnvelopeEnd : "</soap:Envelope>",
	_soapHeaderStart : "<soap:Header>",
	_soapHeaderEnd : "</soap:Header>",
	_soapBodyStart : "<soap:Body", // intentionally unterminated
	_soapBodyEnd : "</soap:Body>"
});

isc.DataSource.addClassMethods({

    create : function (A,B,C,D,E,F,G,H,I,J,K,L,M) {
        var id = A ? A.ID : null;
        if (isc.DataSource._doNotClobber) {
            var instance;
            if (id) instance = isc.DataSource.get(id);
            if (instance) return instance;
        }
        
        if (this.DS_FN) delete this.DS_FN[id];

        return this.invokeSuper(this, "create", A,B,C,D,E,F,G,H,I,J,K,L,M);
    }, 

    // Looking up and loading DataSources
	// --------------------------------------------------------------------------------------------

    isLoaded : function (name) {
		// if no name was passed to us, return false
		if (!name) return false;
        if (isc.isA.DataSource(name) || this._dataSources[name]) return true;
        return false;
    },
    
    //> @classMethod DataSource.getDataSource()
    // Lookup a DataSource by ID.
    //
    // @param ID (identifier) DataSource ID
    // @return (DataSource) the DataSource with this ID, if loaded, otherwise null.
    // @visibility external
    //<
    // If the dataSource does not exist locally and a callback is provided to this method, we
    // attempt to fetch the named datasource from the server and callback.  
    _$refColon:"ref:",
	getDataSource : function (name, callback, context, schemaType) {
		// if no name was passed to us, return null
		if (!name) return null;

//        if (isc.isA.DataSource(name)) {
//            alert("getting datasource: " + name);
//        } else {
//            alert("getting name: " + name);
//        }

        // handle being passed a DataSource instance
        if (isc.isA.DataSource(name)) return name;

        // strip off the "ref:" prefix used in the XML format.  Happens when a field of
        // DataSource type has been declared as an XML attribute (if it's not an attribute you
        // get <DataSource ref="dsId"/> and this is handled during XML->JS translation)
        if (isc.startsWith(name, this._$refColon)) {
            name = name.substring(4);
        }

        if (schemaType && isc.WebService) {
            if (schemaType == isc.DS._$element) return this._elements[name];
            if (schemaType == isc.DS._$type) return this._types[name];
            return null;
        }

		// load the ds (clientside)
        var ds = this._dataSources[name];
		if (!ds) {
            // If this is a DataSource that has been registered as having been created in 
            // SmartGWT, create it JS-side now
            if (isc.DataSource.DS_FN && isc.DataSource.DS_FN[name]) {
                // Remove it from the SGWT pending list before calling into the create 
                // method, otherwise we get a recursion loop
                var work = isc.DataSource.DS_FN[name];
                delete isc.DataSource.DS_FN[name];
                work.call(null);
            }
            ds = this._dataSources[name];
            if (!ds) {
                ds = this._loadDataSource(name, callback);
                if (ds) ds.ID = name; // tell the dataSource its name for later
            }
		}

		if (ds) {
            // if they've specified a callback, call the callback in addition to returning the ds
            if (callback) {
                this.fireCallback(callback, "ds", [ds], ds);
            }
            
            return ds;
		}

		// load from server if a callback was passed
		if (callback) {
            this.loadSchema(name, callback, context);
        }
			
		// let the caller know that we don't have the datasource.  if a callback was passed,
		// the caller will be notified via the callback.
		return null;
	},
    
    // loadSchema - attempt to load a remote dataSource schema from the server.
    // This is supported as part of the SmartClient server functionality
    loadSchema : function (name, callback, context) {
        this.logWarn("Attempt to load schema for DataSource '" + name + 
            "'. This dataSource cannot be found. To load DataSources from the server without " +
            "explicit inclusion in your application requires optional SmartClient server " +
            "support - not present in this build.");
        return null;
    },
    
    
    //> @classMethod DataSource.get()
    // Synonym of +link{getDataSource()}: Lookup a DataSource by ID.
    //
    // @param ID (identifier) DataSource ID
    // @return (DataSource) the DataSource with this ID, if loaded, otherwise null.
    // @visibility external
    //<
    get : function (name, callback, context, schemaType) { 
        return this.getDataSource(name, callback, context, schemaType); 
    },

    _loadDataSource : function (name, callback) {
		if (callback) return null; // no error if we're going to try to load from server.

		//>DEBUG
        if (name != isc.auto && this.logIsDebugEnabled()) {
            this.logDebug("isc.DataSource '" + name + "' not present");
        }
		//<DEBUG

		// couldn't load the dataSource -- return null
		return null;
	},

    // returns a list of all registered datasources
    getRegisteredDataSources : function () {
        return isc.getKeys(this._dataSources);
    },

	// returns true if the named DS is registered, false otherwise
	isRegistered : function (name) {
		if (this._dataSources[name]) return true;
		return false;
	},


    // Foreign Key relationships
	// --------------------------------------------------------------------------------------------
    // field.foreignKey can be [dataSourceID].[fieldName] or just [dataSourceID], where the latter
    // form means it's a foreignKey on the other dataSource's primary key fields.
    
    // methods to break up field.foreignKey into the target DataSource and field name.  Factored in
    // case we ever want to change our somewhat hokey foreign key scheme
    getForeignFieldName : function (field) {
        var foreignKey = field.foreignKey,
            dotIndex = foreignKey.indexOf(".");
    
        // If there is no dot the foreignkey is a field within this dataSource.
        
        if (dotIndex == -1) return foreignKey;
        return foreignKey.substring(dotIndex + 1);
    },

    getForeignDSName : function (field, defaultDS) {
        var foreignKey = field.foreignKey,
            dotIndex = foreignKey.indexOf(".");

        // If there is no dot the foreignkey is a field within this dataSource.
                
        if (dotIndex == -1) {
            if (defaultDS == null) return null;
            return isc.isA.String(defaultDS) ? defaultDS : defaultDS.ID;
        }
        return foreignKey.substring(0, dotIndex);
    },
    
    registerDataSource : function (dataSource) {
        //>DEBUG
        if (this.logIsInfoEnabled()) {
            this.logInfo("Registered new isc.DataSource '" + dataSource.ID + "'");
        }
        //<DEBUG
        if (dataSource.ID) {
            var existingDS = this._dataSources[dataSource.ID];
            // overwrite any existing registered DataSource, unless we have a schemaNamespace.
            // This avoids XMLSchema-derived Schema cloberring ordinary DataSources, since
            // XMLSchema-derived DataSources can be looked up via other means
            // (SchemaSet.getSchema())
            if (!existingDS || !dataSource.schemaNamespace) {
                this._dataSources[dataSource.ID] = dataSource;
            }
            if (dataSource.tableCode) {
                this._tableCodesToDatasourceIds[dataSource.tableCode] = dataSource.ID;
                var linkedDatasources = this._fkTableCodesToDatasourceIds[dataSource.tableCode];
                if (linkedDatasources) {
                    for (var i = 0; i < linkedDatasources.length; i++) {
                        var linkedDSName = linkedDatasources[i];
                        var linkedDS = isc.DS.get(linkedDSName);
                        var linkedDSFields = linkedDS.getFields();
                        var dsFields = dataSource.getFields();
                        for (var linkedDSFieldName in linkedDSFields) {
                            for (var dsFieldName in dsFields) {
                                if (linkedDSFields[linkedDSFieldName].fkColumnCode == dsFields[dsFieldName].columnCode) {
                                    linkedDSFields[linkedDSFieldName].foreignKey = dataSource.ID + "." + dsFields[dsFieldName].name;
                                    break;
                                }
                            }
                        }
                        linkedDS.addChildDataSource(dataSource);
                        linkedDatasources.removeAt(i);
                        i--;
                    }
                    if (linkedDatasources.length == 0) {
                        delete this._fkTableCodesToDatasourceIds[dataSource.tableCode];
                    }
                }
            }
        }

        
        if (isc.Schema && isc.isA.Schema(dataSource)) {
            if (isc.isAn.XSElement(dataSource)) this._elements[dataSource.ID] = dataSource;
            else if (isc.isAn.XSComplexType(dataSource)) this._types[dataSource.ID] = dataSource;
            return;
        }

        
   
        // link DataSources with foreignKeys to each other as they are loaded
        // -----------------------------------------------------------------------------------
        
        var fields = dataSource.getLocalFields(true);

        var unlinkedChildren = this._unlinkedChildren = (this._unlinkedChildren || {});
 
        // try to satisfy this DS's foreignKey relationships
		for (var fieldName in fields) { 
			var field = fields[fieldName];
			var targetDSName = null;
			if (field.fkTableCode) {
			    targetDSName = this._tableCodesToDatasourceIds[field.fkTableCode];
			    // check whether sql datasource with encoded tableName was already loaded
			    if (targetDSName == null) {
			        if (this._fkTableCodesToDatasourceIds[field.fkTableCode] == null) {
			            this._fkTableCodesToDatasourceIds[field.fkTableCode] = [];
			        }
			        this._fkTableCodesToDatasourceIds[field.fkTableCode].add(dataSource.ID);
			        continue;
			    }
			    var ds = isc.DS.get(targetDSName);
                var dsFields = ds.getFields();
                for (var dsFieldName in dsFields) {
                    if (dsFields[dsFieldName].columnCode == field.fkColumnCode) {
                        field.foreignKey = ds.ID + "." + dsFields[dsFieldName].name;
                    }
                }
			} else if (field.foreignKey) {
	            var targetDSName = this.getForeignDSName(field, dataSource);
			} else {
			    continue;
			}
            if (isc.DS.isRegistered(targetDSName)) {
                //this.logWarn("dataSource: " + dataSource.ID + " found parent: " + targetDS.ID);
                // tell the other DataSource that this DS links to it in a 1 to many
                // relationship
                isc.DS.get(targetDSName).addChildDataSource(dataSource);
            } else {
                //this.logWarn("dataSource: " + dataSource.ID + " could not find parent: " +
                //             targetDSName);  
                // add ourselves to the list of future children of targetDS, to be linked up
                // whenever it gets loaded
                if (unlinkedChildren[targetDSName] == null) {
                    unlinkedChildren[targetDSName] = [];
                }
                unlinkedChildren[targetDSName].add(dataSource);
            }
        }

        // see if this new DS satisfies any foreignKey relationships
        var children = unlinkedChildren[dataSource.ID];
        if (children != null) {
            //this.logWarn("dataSource: " + dataSource.ID + " found children: " + children);  
            dataSource.map("addChildDataSource", children);
            unlinkedChildren[dataSource.ID] = null;
        }

        // Link DataSources that declare childRelations to their children
        // ---------------------------------------------------------------------------------
        var unfoundChildren = this._unfoundChildren = this._unfoundChildren || {};
        if (dataSource.childRelations) {
            for (var i = 0; i < dataSource.childRelations.length; i++) {
                var relation = dataSource.childRelations[i],
                    childDSName = relation.dsName,
                    childDS = isc.DS.get(childDSName);
                if (childDS) {
                    //this.logWarn("dataSource: " + dataSource.ID + 
                    //             " found child: " + childDS.ID);
                    this._addChildRelation(dataSource, childDS, relation);
                } else {
                    //this.logWarn("dataSource: " + dataSource.ID + 
                    //             " could not find child: " + childDSName);  
                    if (unfoundChildren[childDSName] == null) {
                        unfoundChildren[childDSName] = [];
                    }
                    relation.parentDS = dataSource.ID;
                    unfoundChildren[childDSName].add(relation);
                }
            }
        }
        // eg User loaded, now Account is loaded
        var relations = unfoundChildren[dataSource.ID];
        if (relations) {
            for (var i = 0; i < relations.length; i++) {
                var relation = relations[i];
                this._addChildRelation(isc.DS.get(relation.parentDS), dataSource, relation);
            }
        }
        
    },

    _addChildRelation : function (parentDS, childDS, relation) {
        parentDS.addChildDataSource(childDS);

        // create foreignKey marker on field
        if (!relation.fieldName) return;

        var field = childDS.getField(relation.fieldName);
        if (!field.foriegnKey) {
            // HACK allows getTreeRelationship to work
            field.foreignKey = parentDS.ID + "." + 
                                    parentDS.getPrimaryKeyFieldNames()[0];
        }
    },

    // SmartGWT DataSource registration
    // ---------------------------------------------------------------------------------------
    
    registerSgwtDS : function (id, commandObject) {
        isc.DataSource.DS_FN[id] = commandObject;
    },
    unregisterSgwtDS : function (id) {
        delete isc.DataSource.DS_FN[id];
    },
    reregisterSgwtDS : function (oldId, newId) {
        isc.DataSource.DS_FN[newId] = isc.DataSource.DS_FN[oldId];
        delete isc.DataSource.DS_FN[oldId];
    },

    // Schema and types
    // ---------------------------------------------------------------------------------------
    
    // Simple helper to get a field value for a record, respecting dataPath and SimpleType.
    //> @classMethod DataSource.getFieldValue()
	// Given a field definition and a record object, this method
	// will return the field value for the record.
	// <P>
	// This method will follow any +link{listGridField.dataPath,dataPath} specified on the 
	// component field if necessary, and will extract 
	// +link{simpleType.getAtomicValue(),atomic values} from custom +link{SimpleType} fields
	// where this is required.
	// @param field (DataSourceField || ListGridField || DetailViewerField || FormItem)
	//  Field definition from a dataSource or dataBoundComponent.
	// @param record (Record) data object
	// @return (any) extracted field value
	// @visibility external
	//<
	getFieldValue : function (field, record) {
		return isc.Canvas._getFieldValue(null, field, record);
	},
    
    // helper method for getObjectField; returns inheritance distance or -1 on error
    getInheritanceDistance : function (superclass, subclass) {
        // check that the arguments are in fact classes, and confirm their relationship
        var superclassObj = isc.ClassFactory.getClass(superclass),
            subclassObj = isc.ClassFactory.getClass(subclass)
        ;
        if (superclassObj == null || subclassObj == null) {
            this.logWarn("Invalid superclass and/or subclass argument provided");
            return -1;
        }
        if (!subclassObj.isA(superclass)) {
            this.logWarn(subclass + " is not a subclass of " + superclass);
            return -1;
        }
        
        for (var distance = 0; subclassObj != superclassObj; distance++) {
            subclassObj = subclassObj.getSuperClass();
        }
        
        return distance;
    },

    // simple: String, Number, Boolean, Date, Time, Function
    // non-simple: null, Object, Array
    
    isSimpleTypeValue : function (value) {
        if (value != null && (!isc.isAn.Object(value) || isc.isA.Date(value))) return true;
        return false;
    },

    // Given an object, find the nearest schema to be used to serialize it.
    // If there is a SmartClient Class, but no schema exists (eg custom class), we find the
    // nearest SmartClient super class that has a schema.
    getNearestSchema : function (object) {
        if (object == null) return null;
    
        var className;
        if (isc.isA.String(object)) className = object;
        else {
            className = isc.isAn.Instance(object) ? object.getClassName() : 
                                                    object._constructor || object.type || object.$schemaId;
        }

		var schema = isc.DS.get(className);
		
        // if no schema is loaded for this class, look for a superclass schema
		var theClass = isc.ClassFactory.getClass(className);

        if (theClass != null) {
    		var lastSuperClass = null;
	    	while (schema == null && (theClass = theClass.getSuperClass()) != null && 
                                      theClass != lastSuperClass) 
            {
	    		schema = isc.DS.get(theClass.getClassName());
		    	lastSuperClass = theClass;
    		}
        }
        return schema || isc.DS.get("Object");
    },
    
    // Given a SmartClient component schema name, determine the associated SmartClient class
    
    getNearestSchemaClass : function (schema) {
        if (schema == null) return null;
        var theClass;
        
        while (theClass == null) {
            // Pick up the dataSource for the className passed in.
            // This will allow us to determine the correct SmartClient class for the object
            // by iterating up the datasource parent chain if necessary
            var schema = isc.DS.get(schema);
            // Catch the case where we're passed a schema string that we don't recognize
            if (schema == null) return null;
            theClass = isc.ClassFactory.getClass(schema._constructor || schema.Constructor 
                                                                     || schema.type);
            //this.logWarn("schema is: " + schema + ", theClass: " + theClass + 
            //             ", schema.inheritsFrom: " + schema.inheritsFrom);
            
            if (theClass != null) return theClass;
            schema = schema.inheritsFrom;
            
            if (!schema) return null;
        }
        return null;
    },

    // Standard Operations
	// ----------------------------------------------------------------------------------------

    // Centrally handle backcompat synonyms for operation types
    _getStandardOperationType : function (operationType) {
        switch (operationType) {
            case "fetch":
            case "select":
            case "filter":
                return "fetch";
            case "add":
            case "insert":
                return "add";
            case "update": 
                return "update";
            case "remove":
            case "delete":
                return "remove";

            default:
                // cases not covered:
                // "replace": no longer supported, but not equivalent to any supported type
                // "validate": never had a synonym

            return operationType;
        }
    },

	isClientOnly : function (dataSource) {
		if (isc.isA.String(dataSource)) dataSource = this.getDataSource(dataSource);
		if (!dataSource) return false;
		return dataSource.clientOnly;
	},

	// ResultSets creation
	// --------------------------------------------------------------------------------------------

    // make a default operation for the specified dataSource, using the specified application.
	// if no application is specified, use the default application.
    makeDefaultOperation : function (dataSourceId, operationType, operationId) 
    {
		// XXX this is really only intended for the standard set of operations (fetch, update,
        // add, remove): check that operationType is valid?
		
		var theApp = isc.rpc.app();

        if (isc.isA.DataSource(dataSourceId)) dataSourceId = dataSourceId.ID;
        if (!dataSourceId) {
            dataSourceId = "auto";
        } else if (operationId) {
            
            // If we have a valid dataSource and operationId, store/lookup against the 
            // dataSource, not the app - so, eg, we can have an operationId "customFetch" 
            // on more than one dataSource
            var dataSource = isc.DataSource.get(dataSourceId);
            if (isc.isA.DataSource(dataSource)) {
                if (!dataSource.createdOperations) dataSource.createdOperations = {};
                var operation = dataSource.createdOperations[operationId];
                if (operation == null) {
                    operation = {ID:operationId,
                                 dataSource:dataSourceId,
                                 type:operationType, 
                                 filterType:"paged",
                                 loadDataOnDemand:true};
                    dataSource.createdOperations[operationId] = operation;
                }
                return operation;
            } 
        }
    
        // NOTE: removed code that added the datasource to the application if it wasn't
        // already there so that it could be looked up when performing the operation;
        // now behavior is that if no datasources are defined on the app, it will just
        // look for a currently loaded datasource with that name

        // make sure the app has an operations array
        if (theApp.operations == null) theApp.operations = {};

        // create the default operation on this DataSource

        // allow an operationId, otherwise create one based on DS and operation type
        operationId = operationId || dataSourceId + "_" + operationType;
        var operation = theApp.operations[operationId];
        if (operation == null) {
            operation = {ID:operationId,
                         dataSource:dataSourceId,
                         type:operationType, 
                         filterType:"paged",
                         loadDataOnDemand:true,
                         source:"auto"};
            theApp.operations[operationId] = operation;

            
        }

        // return the operation
        return operation;
    },

	// ResultSet cache sync
	// --------------------------------------------------------------------------------------------
    // NOTE: if an operation modifies more than one DataSource, obviously its not going to make
    // sense to pass the same updateData to ResultSets on different DataSources.  We assume you
    // know that you don't have a ResultSet on the second DataSource, or are dropping cache on
    // updates!

    handleUpdate : function (dsResponse, dsRequest) { 
        if (!this.isUpdateOperation(dsRequest.operationType)) return;
        
        // notify the datasource that the operation affects
        var ds = this.get(dsRequest.dataSource);
        ds.updateCaches(dsResponse, dsRequest);
    },

	// is this an operation which alters data?
	isUpdateOperation : function (operation) {
		if (operation == "add" || operation == "update" || operation == "remove" || 
            operation == "replace" 
            //>!BackCompat 2004.7.23 synonyms
            || operation == "delete" || operation == "insert" 
            //<!BackCompat
           ) return true;
	},
    
    getUpdatedData : function (dsRequest, dsResponse, useDataFromRequest) {
        var ds = this.get(dsRequest.dataSource);
        return ds.getUpdatedData(dsRequest, dsResponse, useDataFromRequest);
    },

    // ---------------------------------------------------------------------------------------
    
	// given a set of values from a DynamicForm, return appropriate filter criteria
	filterCriteriaForFormValues : function (formValues) {
        if (isc.DS.isAdvancedCriteria(formValues)) return formValues;
    
		var filterValues = {};
		for (var fieldName in formValues) {
			var formValue = formValues[fieldName];
			// Form fields may have the value "" if they held a value and were cleared out.  A
            // blank field in a form used as a filter means "field can have any value", not
            // "field must be blank", so we eliminate blank field->value pairs.
			// This also goes for a selectList where a blank value is selected, which is
            // supposed to mean "Any".
			if (formValue == null || isc.is.emptyString(formValue)) continue;

			// multi-selects are returned as an array.  
			if (isc.isAn.Array(formValue)) {
                
                // If nothing is selected we get [], meaning no criteria
                if (formValue.length == 0) continue;
			    // If blank is selected, we get ["", "otherValue", ...], meaning "field can
                // have any value".
                
                for (var i = 0; i < formValue.length; i++) {
                    var orValue = formValue[i];
                    if (isc.isAn.emptyString(orValue)) continue;
                }
            }
			
			// use the formValue as is
			filterValues[fieldName] = formValue;

			// NOTE: we do this on the client rather than the server because otherwise the client
			// would be unable to express "field must be blank" - the server would discard the
			// criterion assuming it was an artifact of the filterValues having come from a
			// DynamicForm.
		}
		return filterValues;
	},

    checkEmptyCriteria : function (criteria, subCriterion) {
    	
        if ((subCriterion || this.isAdvancedCriteria(criteria)) && criteria.criteria) {
            if (criteria.criteria.length == 0 ) return null;

            for (var i=criteria.criteria.length-1; i>=0; i--) {
                var subCrit = criteria.criteria[i],
                    shouldRemove = false;

                if (!subCrit) shouldRemove = true;
                else {
                    if (!subCrit.criteria) {
                        // is this an empty object?
                        if (isc.isA.emptyObject(subCrit)) shouldRemove = true;
                    } else {
                        var crit = this.checkEmptyCriteria(subCrit, true);
                        if (crit) criteria.criteria[i] = crit;
                        else shouldRemove = true;
                    }
                }

                if (shouldRemove) criteria.criteria.removeAt(i);
            }
        }

        return criteria
    },

    //> @classAttr DataSource.serializeTimeAsDatetime (boolean : false : IRA)
    // Specifies how time field values should be serialized when being sent to the server for
    // dataSources with dataFormat <code>"xml"</code> or <code>"json"</code>.
    // If <code>false</code> the time field will be serialized as a logical time object in
    // UTC, using the standard truncated XML Schema format: <code>"HH:MM:SS"</code>.
    // If <code>true</code> the time field will be serialized as a complete dateTime object
    // matching the value of the underlying JavaScript date object representing this time on the
    // client.
    // @visibility external
    //<
    
    serializeTimeAsDatetime:false,
    
    //> @classMethod DataSource.load()
    // Load a DataSource or an array of DataSources using the DataSourceLoader servlet.  When
    // a callback is specified, this is fired after the DataSources are loaded.  The callback
    // is passed a single parameter, the <code>dsID</code> list passed into the method.  If
    // no loading occurs because the requested DataSource(s) are already loaded, a warning is 
    // logged and the callback is fired immediately.
    // <P>
    // To force reloading of DataSources that have already been loaded,
    // pass <code>true</code> for the forceReload parameter.
    //
    // @param dsID (String | Array of String) DataSource ID or Array of DataSource IDs
    // @param callback (Callback) Callback to fire after DataSource loading completes
    // @param [forceReload] (Boolean) Forcibly reload a dataSource if it's already loaded
    // @visibility external
    //<
    load : function (dsID, callback, forceReload, loadParents) {
        if (!isc.isAn.Array(dsID)) dsID = [dsID];

        if (dsID.length <= 0) {
            this.logWarn("No DataSource IDs passed in.");
            return;
        }

        var loadThese = [];
        
        for (var i=0; i<dsID.length; i++) {
            if (!this.isLoaded(dsID[i]) || forceReload) loadThese.add(dsID[i]);
        }

        var dsList = loadThese.join(","),
            url = isc.DataSource.loaderURL + "?" +
                    (loadParents ? "loadParents=true&" : "") + "dataSource="+dsList,
            _dsID = dsID;
        ;

        if (loadThese.length > 0) {
            // post out to the DataSourceLoader servlet
            isc.RPCManager.send(null, 
                function (rpcResponse, data, rpcRequest) {
                    //!OBFUSCATEOK
                    if (rpcResponse.httpResponseCode == 404) {
                        isc.warn("The DataSourceLoader servlet is not installed.");
                        return null;
                    }
                    if (loadParents && !forceReload) isc.DataSource._doNotClobber = true;
                    try {
                        // eval the returned create() statement(s) and fire the callback if passed
                        eval(data);
                    } catch(e) {
                        isc.logWarn("Encountered problems trying to load DataSources: " + 
                                            isc.echoAll(e));
                    }
                    delete isc.DataSource._doNotClobber;
                    if (callback) this.fireCallback(callback, ["dsID"], [_dsID]);
                },
                {
                    actionURL: url,
                    httpMethod: "GET",
                    willHandleError: true
                }
            );
        } else {
            this.logWarn("DataSource(s) already loaded: " + dsID.join(",") + 
                "\nUse forceReload to reload such DataSources");
            if (callback) this.fireCallback(callback, ["dsID"], [_dsID]);
        }
    },
    
    //> @classMethod dataSource.loadWithParents()
    // Variation of +link{DataSource.load()} that will also automatically load any DataSources that
    // the requested DataSources inherit from (via +link{DataSource.inheritsFrom}).
    // <P>
    // If the parent DataSource is already loaded, calling <code>loadWithParents</code> will not
    // automatically reload them unless the forceReload parameter is passed.
    // 
    // @param dsID (String | Array of String) DataSource ID or Array of DataSource IDs
    // @param callback (Callback) Callback to fire after DataSource loading completes
    // @param [forceReload] (Boolean) Forcibly reload a dataSource if it's already loaded
    //
    // @visibility external
    //<
    loadWithParents : function (dsID, callback, forceReload) {
        this.load(dsID, callback, forceReload, true);
    },
    
    //> @classMethod DataSource.getSortBy()
    // Given an array of +link{SortSpecifier}s, return a simple list of Strings in the format
    // expected by +link{dsRequest.sortBy}.
    //
    // @param sortSpecifiers (Array of SortSpecifier) The list of specifiers to return in  
    // sortBy format
    // @return (Array of String) An array of sort-definitions in the format expected by 
    // +link{dsRequest.sortBy}
    // @visibility external
    //<
    getSortBy : function (sortSpecifiers) {
        if (!isc.isA.Array(sortSpecifiers)) sortSpecifiers = [sortSpecifiers];
        
        var sortArray = [];
        for (var i=0; i<sortSpecifiers.length; i++) {
            var item=sortSpecifiers.get(i);
            sortArray.add((!Array.shouldSortAscending(item.direction) ? "-" : "") + item.property);
        }
        return sortArray;
    },

    //> @classMethod DataSource.getSortSpecifiers()
    // Return a an array of +link{SortSpecifier}s, given an array of Strings in the format
    // expected by +link{dsRequest.sortBy}.
    //
    // @param sortBy (Array of String) A list of sortBy strings in the format expected by +link{dsRequest.sortBy}
    // @return (Array of String) An array of +link{SortSpecifier}s equivalent to the passed in string array
    // @visibility external
    //<
    getSortSpecifiers : function (sortBy) {
        if (!isc.isA.Array(sortBy)) sortBy = [sortBy];
        
        var specifierArray = [];
        for (var i=0; i<sortBy.length; i++) {
            var item = sortBy.get(i),
                direction = "ascending",
                property = item
            ;
            if (item.substring(0,1) == "-") {
                direction = "descending";
                property = item.substring(1);
            }
            specifierArray.add({ property: property, direction: direction})
        }
        return specifierArray;
    },

    // class-level version of isAdvancedCriteria - accepts a DS as a param, passed in by calls
    // from the instance-level method
    isAdvancedCriteria : function (criteria, dataSource) {
        if (!criteria) return false;

        if (!dataSource) {
            // Is it explcitily marked as AdvancedCiteria?
            return (criteria && criteria._constructor == "AdvancedCriteria");
        }

        if (!isc.isA.DataSource(dataSource)) dataSource = this.get(dataSource);

        // we have a dataSource
        if (criteria._constructor == "AdvancedCriteria") return true;
        
        // Not explicitly marked, so we'll make a guess.
        var returnVal = false;

        // We'll assume it's AdvancedCriteria if the fieldName property refers to a valid
        // field on this DS, and the operator property refers to a valid operator.  This
        // would be extremely unlikely to occur even if a DataSource existed to store criteria
        // in fieldName/operator/value format, since Records for such a DataSource would not
        // have values for "fieldName" that would equal the fieldNames of such a DataSource (eg
        // "fieldName", "operator" or "value")
        if (dataSource.getField(criteria.fieldName) &&
            dataSource.getSearchOperator(criteria.operator))
        {
            returnVal = true;
        }
 
        // Before applying further heuristics, make sure that this DataSource
        // doesn't have any fields that are actually called "fieldName" or "operator"
        else if (dataSource.getField("fieldName") || dataSource.getField("operator")) returnVal = false;
 
        // if the criteria has fieldName and operator attribues *AND* the operator is
        // valid, assume we have advancedCriteria - this caters for criteria agains calculated
        // fields, which do not appear as DS-fields
        
        else if (criteria.fieldName && 
                (criteria.operator && dataSource.getSearchOperator(criteria.operator))) {
            returnVal = true;
        }

        else {
            // We'll also assume it's an AdvancedCriteria if there is no fieldName property and
            // the operator is either "and" or "or".
            var undef;
            if (criteria.operator != undef) {
                var op = dataSource.getSearchOperator(criteria.operator);
                if (op != null && (op.isAnd || op.isOr)) {
                    returnVal = true;
                }
            }
        }
        this.logInfo("Criteria object:" + this.echo(criteria) + " not explicitly marked as AdvancedCriteria" + 
            (returnVal ? " but treating as Advanced based on format." : " - treating as SimpleCriteria."), "AdvancedCriteria");
        return returnVal;
    },

    //> @classMethod DataSource.copyCriteria
    // Create a copy of a criteria.
    //
    // @param criteria (Criteria) criteria to copy
    // @return (Criteria) copy of criteria
    // @visibility external
    //<
    copyCriteria : function (criteria) {
        if (criteria == null) return null;
        return isc.clone(criteria);
    },
    
    getCriteriaFields : function (criteria, dataSource, isSubcriterion) {
        if (dataSource && !isc.isA.DataSource(dataSource)) dataSource = this.get(dataSource);
        if (isSubcriterion || 
            (dataSource ? dataSource.isAdvancedCriteria(criteria) : 
                this.isAdvancedCriteria(criteria))
           ) 
        {
            var fieldNames = [];
            this._getAdvancedCriteriaFields (criteria, fieldNames);
            return fieldNames;
        }
        return isc.getKeys(criteria);
    },

    // Drill into advancedCriteria object extracting fields
    _getAdvancedCriteriaFields : function (criterion, fieldNames) {
        if (criterion.criteria) {
            for (var i = 0; i < criterion.criteria.length; i++) {
                isc.DS._getAdvancedCriteriaFields(criterion.criteria[i], fieldNames);
            }
        } else {
            fieldNames.add(criterion.fieldName);
        }
    },


    // ---------- LOCAL RESPONSE CACHING ----------
    _responseCache: [],
    maxResponsesToCache: 100,
    cacheResponse : function (dsRequest, dsResponse) {
        if (isc.Offline) {
            var requestKey = isc.Offline.serialize(isc.Offline.trimRequest(dsRequest));
            var index = this._responseCache.findIndex("requestKey", requestKey);
            if (index != -1) {
                this._responseCache.set(index, dsResponse);
            } else {
                if (this._responseCache.length >= this.maxResponsesToCache) {
                    this._responseCache.removeAt(0);
                }
                this._responseCache.add({
                    requestKey: requestKey,
                    dsResponse: dsResponse
                });
            }
        }
    },
    
    getCachedResponse : function (dsRequest) {
        if (isc.Offline) {
            var requestKey = isc.Offline.serialize(isc.Offline.trimRequest(dsRequest));
            return this._responseCache.find("requestKey", requestKey);
        }
        return null;
    },
    
    
    //> @classAttr DataSource.offlineMessage  (string : "This data not available while offline" : IRW)
    // A message returned by a DataSource when it is returning an empty dataset for a fetch
    // because the browser is currently offline and there is no suitable cached offline response.
    // @group i18nMessages, offlineGroup
    // @visibility external
    //<
    offlineMessage: "This data not available while offline",
	
	//> @classAttr DataSource.maxFileSizeExceededMessage  (string : "Size of '${uploadedFileName}' (${uploadedFileSize}) exceeded maximum allowed file size of ${maxFileSize}" : IRW)
    // A message returned by a DataSource when an uploaded file exceeds dataSourceField.maxFileSize
    // @group i18nMessages
    // @visibility external
    //<
    maxFileSizeExceededMessage: "Size of '${uploadedFileName}' (${uploadedFileSize}) exceeded maximum allowed file size of ${maxFileSize}",
    
    
    getPathValue : function (record, fieldPath, field) {
    	//var field;
    	//if (isc.isAn.Object(fieldPath)) {
        //	field = fieldPath;
    	//	fieldPath = null;
    	//}
        return isc.Canvas._getFieldValue(fieldPath, field, record, null, true);
    },
    
    // setPathValue - corollary for getPathValue()
    setPathValue : function (record, fieldPath, field, value) {
    	//var field;
    	//if (isc.isAn.Object(field)) {
    	//	field = fieldPath;
    	//	fieldPath = null;
		//}
        isc.Canvas._saveFieldValue(fieldPath, field, value, record, null, true);
    },
    
    // Given an array of dataSources, return field identifiers for all fields in the dataSources
    // These will be of the form "dataSourceID.fieldName"
    getCombinedDataSourceFields : function (dataSources) {
        var fields = [];
        for (var i = 0; i < dataSources.length; i++) {
            var dataSource = dataSources[i];
            if (isc.isA.String(dataSource)) dataSource = isc.DataSource.get(dataSource);
            if (dataSource == null) {
                this.logWarn("getCombinedDataSourceFields() - unable to locate dataSource:"
                    + dataSources[i]);
                continue;
            }
            var dsID = dataSource.getID(),
                dsFields = isc.getKeys(dataSource.getFields());
            for (var j = 0; j < dsFields.length; j++) {
                fields[fields.length] = dsID + "." + dsFields[j];
            }
        }
        return fields;
    },
    
    // Given a ds field identifier like "dataSourceId.fieldName", extract the field definition
    // from the appropriate dataSource.
    // The dataSources array is optional - if present, pull from that set of dataSources
    // rather than doing a standard isc.DS.get()
    
    
    getFieldFromDataSources : function (combinedFieldId, dataSources, suppressWarning) {
        var dataSource = this.getDataSourceForField(combinedFieldId, dataSources, suppressWarning);
        if (dataSource == null) {
            if (!suppressWarning) {
            	this.logWarn("getFieldFromDataSources() - unable to find dataSource " +
                	" from combined field ID " + combinedFieldId);
			}
            return null;
        }
        
        var splitFieldId = combinedFieldId.split("."),
            dsId = splitFieldId[0],
            dsFieldName = splitFieldId[1];
        return dataSource.getField(dsFieldName);
        
    },
    
    getDataSourceForField : function (combinedFieldId, dataSources, suppressWarning) {

        var splitFieldId = combinedFieldId.split("."),
            dsId = splitFieldId[0],
            dsFieldName = splitFieldId[1];
        if (dsId == null || dsFieldName == null) {
            if (!suppressWarning) {
            	this.logWarn("getDataSourceForField() passed invalid field ID:" + combinedFieldId +
					". Expected format is to use dot-notation to indicate the dataSource plus " +
					"fieldName - for example 'countryDS.countryCode'");
			}
            return null;
        }
        var dataSource;
        if (dataSources == null) dataSource = isc.DataSource.get(dsId);
        else {
            // sanity check only
            if (!isc.isAn.Array(dataSources)) dataSources = [dataSources];
            for (var i = 0; i < dataSources.length; i++) {
                if (dataSources[i].getID() == dsId) {
                    dataSource = dataSources[i];
                    break;
                }
            }
        }
        return dataSource;
    },

    _getDefaultValueForFieldType : function (fieldType) {
        var value;
        switch (fieldType) {
            case "date":
            case "datetime":
            case "time":
                value = new Date(0);
                break;
            case "float":
                value = 0.0;
                break;
            case "integer":
                value = 0;
                break;
            case "boolean":
                value = false;
                break;
            case "text":
            default:
                value = "";
                break;
        }
        return value;
    },
                                 
    _getTypedValueAsString : function (value, fieldType, settings) {
        if (settings == null) settings = {};

        if (value == null) {
            var defaultText = settings.nullValueText;
            if (defaultText != null) return defaultText;
            // if textExportSettings.nullValueText is null, then
            // set value to the default value for the field's type
            value = this._getDefaultValueForFieldType(fieldType);
        }
        switch (fieldType) {
            case "date":
                var format = settings.dateFormat;
                return value.toShortDate(format);
            case "datetime":
                var format = settings.dateTimeFormat;
                return value.toShortDateTime(format);
            case "time":
                var format = settings.timeFormat;
                return isc.Time.toShortTime(value, format);
            case "float":
            case "integer":
            case "boolean":
            default:
                return value.toString();
            case "text":
                return value;
        }
    }
    
	

    
});


isc.DataSource.addProperties({
    //> @attr dataSource.ID              (string : null : [IR])
	// Unique identifier for this DataSource.  Required for all DataSources.
    //
    // DataSources will make themselves available as JavaScript globals under the same name
    // as their ID only if <code>addGlobalId</code> is set.
    //
    // @group identity
    // @serverDS allowed
    // @visibility external
    //<

    //> @attr dataSource.addGlobalId    (boolean : true : IRA)
    // Whether to make this DataSource available as a global variable for convenience.
    // @group identity
    // @visibility external
    //<
    addGlobalId:true,

    //> @attr dataSource.dataFormat (DSDataFormat : "iscServer" : [IR])
    // Indicates the format to be used for HTTP requests and responses when
    // fulfilling DSRequests (eg, when +link{DataSource.fetchData()} is called).
    //
    // @group clientDataIntegration
    // @group serverDataIntegration
    // @serverDS allowed
    // @visibility external
    // @example jsonDataSource
    // @example simpleJSON
    //<
    dataFormat:"iscServer",
    
    //> @attr dataSource.useStrictJSON (boolean : null : IR)
    // Should HTTP responses to requests by this dataSource be formatted using the
    // strict JSON subset of the javascript language?
    // If set to true, responses returned by the server should match the format described
    // +externalLink{http://www.json.org/js.html,here}.
    // <P>
    // Only applies to dataSources which send requests to a server and have
    // +link{dataSource.dataFormat} set to "json" or "iscServer".
    // <P>
    // <b>Note:</b> using strict JSON avoids a known issue in Internet Explorer 9 where
    // datasource transactions can leak memory due to a browser behavior where the native
    // <code>eval()</code> method fails to clean up references when the objects
    // go out of scope. See +link{isc.RPCManager.allowIE9Leak} for more on this.
    // @visibility external
    //<
    
    //> @attr dsRequest.useStrictJSON (boolean : null : IR)
    // Should the HTTP response to this request be formatted using the
    // strict JSON subset of the javascript language?
    // If set to true, responses returned by the server should match the format described
    // +externalLink{http://www.json.org/js.html,here}.
    // <P>
    // Only applies to requests sent a server with
    // +link{dataSource.dataFormat} set to "json" or "iscServer".
    // @visibility external
    //<
    
    

    //> @attr dataSource.dataProtocol (DSProtocol : null : [IR])
    // Controls the format in which inputs are sent to the dataURL when fulfilling DSRequests.
    // May be overridden for individual request types using
    // +link{OperationBinding.dataProtocol,operation bindings}.
    // @group clientDataIntegration
    // @group serverDataIntegration
    // @serverDS allowed
    // @visibility external
    //<

    //> @attr dataSource.useHttpProxy (boolean : null : IR)
    // Like +link{operationBinding.useHttpProxy}, but serves as a default for this DataSource
    // that may be overridden by individual operationBindings.
    //
    // @group clientDataIntegration
    // @visibility external
    //<
    
    //> @attr dataSource.callbackParam (String : "callback" : [IR])
    // Applies only to dataFormat: "json" and +link{dataTransport}:"scriptInclude".  Specifies
    // the name of the query parameter that tells your JSON service what function to call as
    // part of the response.
    //
    // @see dataSource.dataFormat
    // @see dataSource.operationBindings
    // @see operationBinding.callbackParam
    // @group clientDataIntegration
    // @visibility external
    // @serverDS allowed
    // @example xmlEditSave
    //<
    callbackParam : "callback",

    //> @attr dataSource.requestProperties (DSRequest Properties : null : [IR])
    // Additional properties to pass through to the +link{DSRequest}s made by this
    // DataSource.
    // <p>
    // These properties are applied before +link{dataSource.transformRequest} is called.
    //
    // @see DSRequest
    // @see operationBinding.requestProperties
    // @group clientDataIntegration
    // @group serverDataIntegration
    // @serverDS allowed
    // @visibility external
    //<

    //> @attr dataSource.dataTransport      (RPCTransport : RPCManager.defaultTransport : [IR])
    // Transport to use for all operations on this DataSource. Defaults to
    // +link{RPCManager.defaultTransport}.  This would typically only be set to
    // enable "scriptInclude" transport for contacting +link{DataSource.dataFormat,JSON} web
    // services hosted on servers other than the origin server.
    // <p>
    // When using the "scriptInclude" transport, be sure to set +link{dataSource.callbackParam} or
    // +link{operationBinding.callbackParam} to match the name of the query parameter name expected
    // by your JSON service provider.
    //
    // @see RPCTransport
    // @see dataSource.callbackParam
    // @group clientDataIntegration
    // @serverDS allowed
    // @visibility xmlBinding
    //<
    
    //> @attr dataSource.preventHTTPCaching (Boolean : true : IR)
    // If set, the DataSource will ensure that it never uses a cached HTTP response, even if the
    // server marks the response as cacheable.
    // <P>
    // Note that this does not disable caching at higher levels in the framework, for example, the
    // caching performed by +link{ResultSet}.
    // @serverDS allowed
    // @visibility external
    //<
    preventHTTPCaching:true,

    //> @type DSDataFormat
    // Indicates the format to be used for HTTP requests and responses when
    // fulfilling DSRequests (eg, when +link{DataSource.fetchData()} is called).
    // <P>
    // Note that +link{RPCManager.startQueue(),request queuing} is only available for
    // "iscServer" requests.
    //
    // @value "iscServer"
    // Make an HTTP request in a format recognized by the ISC server and expect ISC server
    // response format.  +link{group:serverDataIntegration,Server-side integration} can then be
    // used to connect to any kind of data store or web service.
    //
    // @value "xml"
    // Expect XML responses.  Request format depends on the setting for
    // +link{operationBinding.dataProtocol,protocol}.  This is the correct setting when
    // consuming RSS feeds, XML web services (whether SOAP, REST, XML-RPC or custom format),
    // and XML flat files directly from the browser.
    // <P>
    // Values for "date", "time" or "datetime" fields in responses should be specified in the
    // applicable <a target=_blank href="http://www.w3.org/TR/xmlschema-2/#dateTime">XML Schema
    // date format</a>.
    // If no timezone is explicitly specified, dates / datetimes received by the client are 
    // assumed to be GMT. Note that "date" type fields represent logical dates and may omit
    // time information entirely, and "time" type fields may omit date information.
    // See +link{group:dateFormatAndStorage,Date and Time Format and storage} for more information
    // on how date values are serialized in requests sent to the server.
    //  
    // <P>
    // A DSResponse will be derived from the returned XML via the process described under
    // +link{dataSource.transformResponse()}.
    //
    // @value "json"
    // Expect response in JSON +externalLink{http://json.org,(Java Script Object Notation)}
    // format, ready to be eval()'d. Response should either be a naked object literal:<br>
    // <code>&nbsp;&nbsp;&nbsp;&nbsp;{ "property":"value1", "property2" : "value2", ... }</code><br>
    // or a string that evals to return a valid response object:<br>
    // <code>&nbsp;&nbsp;&nbsp;&nbsp;var response = { "property":"value1", "property2" : "value2", ... }</code>
    // <br>
    // <code>&nbsp;&nbsp;&nbsp;&nbsp;response;</code><br>
    // <P>
    // A DSResponse will be derived from the returned JSON via the process described under
    // +link{dataSource.transformResponse()}.
    // <P>
    // As with <code>"xml"</code> responses, values for "date" or "datetime" fields should be
    // specified as a String in 
    // <a target=_blank href="http://www.w3.org/TR/xmlschema-2/#dateTime">XML Schema date format</a>
    // and may include a timezone.  In the absence of a timezone they will be assumed to be GMT.
    // <P>
    // Request format depends on the setting for +link{operationBinding.dataProtocol,protocol}.  
    // See also +link{XJSONDataSource}.
    //
    // @value "custom"
    // SmartClient will not attempt to parse the response, instead,
    // +link{dataSource.transformResponse()} must be implemented.
    // <code>transformResponse</code> will receive the "data" parameter as a String, and must
    // parse this String into an Array of Objects, which should be set as
    // +link{dsResponse.data}.  Request format depends on the setting for
    // +link{operationBinding.dataProtocol,protocol}.
    // <P>
    // Note that, unlike either the "json" or "xml" settings of <code>dataFormat</code>, you
    // are responsible for ensuring that parsed values are the correct type, for example, using
    // the JavaScript built-ins <code>parseInt</code> and <code>parseFloat</code> on integer
    // and decimal values respectively, and using <code>new Date()</code> to construct valid
    // Dates.
    //
    // @group clientDataIntegration
    // @group serverDataIntegration
    // @visibility external
    // @serverDS allowed
    // @example simpleJSON
    //<

    //> @attr dataSource.serverType (DSServerType : "generic" : [IR])
    // For a DataSource stored in .xml format on the SmartClient server, indicates what server-side
    // connector to use to execute requests, that is, what happens if you call
    // dsRequest.execute() in server code.
    // 
    // @group serverDataIntegration
    // @requiresModules SCServer
    // @serverDS only
    // @visibility external
    //<
    
    //> @type DSServerType
    // Indicates what SmartClient Server will do with a DataSource request if you call
    // dsRequest.execute() in server code.
    // <P>
    // If you use a Java-based persistence layer not provided by SmartClient, such as
    // EJB or your own custom object model, you don't need to set
    // <code>dataSource.serverType</code> and should follow the
    // +link{group:serverDataIntegration,integration instructions}.
    //
    // @value "sql"
    // Use SmartClient's +link{group:sqlDataSource,built-in SQL connectors} to talk directly
    // to relational databases.
    // 
    // @value "hibernate"
    // Use Hibernate, either using a real mapped bean or by automatically generating a 
    // Hibernate configuration based on a SmartClient DataSource file 
    // (<i>dataSourceID</i>.ds.xml).  See +link{group:hibernateIntegration} for details.
    //
    // @value "jpa"
    // Use SmartClient's built-in JPA 2.0 connector.
    //
    // @value "jpa1"
    // Use SmartClient's built-in JPA 1.0 connector.
    //
    // @value "generic"
    // Requests will be delivered to the server and you are expected to write Java code to
    // create a valid response.  Throws an error if the server side method dsRequest.execute()
    // is called. This is appropriate if you intend an entirely custom implementation, and you
    // want an error thrown if there is an attempt to call an operation you have not
    // implemented.
    //
    // @group serverDataIntegration
    // @serverDS allowed
    // @visibility external
    //<

    //> @attr dataSource.tableName (String : null : [IR])
    // For DataSources using the +link{group:sqlDataSource,SmartClient SQL engine} for
    // persistence, what database table name to use.  The default is to use the DataSource ID
    // as the table name.
    // 
    // @group serverDataIntegration
    // @requiresModules SCServer
    // @serverDS only
    // @visibility external
    //<
    
    //> @attr dataSource.schema (String : null : [IR])
    // <b>This property only applies to the built-in SQL DataSource provided in Pro and better 
    // editions of SmartClient</b>
    // <p>
    // Defines the name of the schema we use to qualify the +link{tableName,tableName} in generated
    // SQL.  If you do not provide this property, table names will not be qualified in generated 
    // SQL, and thus the default schema will be used.  Support for multiple schemas (or schemata)
    // varies quite significantly across the supported databases, as does the meaning of the 
    // phrase "default schema".  In addition, some databases allow you to override the default 
    // schema in the JDBC connection URL, which is a preferable approach if all your tables are 
    // in the same (non-default) schema.
    // <p>
    // The following table provides information by product:
    // <p>
    // <table border="1" class="normal" width="90%">
    // <tr><td> Product</td><td width="90%"> Notes</td></tr>
    // <tr><td> DB2</td><td>
    // Arbitrarily named schemas are supported.  The default schema is named after the connecting
    // user, though this can be overridden by specifying the "currentSchema" property on the JDBC
    // connection URL</td></tr>
    // <tr><td> DB2 for iSeries</td><td>
    // Arbitrarily named schemas are supported.  "Schema" is synonymous with "library".  The 
    // default schema depends on the setting of the "naming" connection property.  When this is 
    // set to "sql", behavior is similar to other DB2 editions: the default schema is named after
    // the connecting user, unless overridden by specifying a library name in the JDBC connection 
    // URL.  When "naming" is set to "system", the schema of an unqualified table is resolved 
    // using a traditional search of the library list; the library list can be provided in the 
    // "libraries" property</td></tr>
    // <tr><td> Firebird</td><td>
    // Firebird does not support the concept of schema at all - all "schema objects" like tables 
    // and indexes belong directly to the database.  In addition, Firebird actively rejects 
    // qualified table names in queries as syntax errors; therefore, you should <b>not</b> set
    // the <code>schema</code> property for a DataSource that will be backed by a Firebird database
    // </td></tr>
    // <tr><td> HSQLDB</td><td>
    // Arbitrarily named schemas are supported.  The default schema is auto-created when the
    // database is created; by default it is called "PUBLIC", but can be renamed.  It is not 
    // possible to set the default schema in the JDBC connection URL</td></tr>
    // <tr><td> Informix</td><td>
    // Informix databases can be flagged as "ANSI mode" at creation time.  ANSI-mode databases 
    // behave similarly to DB2 for schema support: arbitrarily named schemas are supported, and
    // the default schema is the one named after the connected user.  Non-ANSI databases have no
    // real schema support at all. It is not possible to set the default schema in the JDBC 
    // connection URL with either type of database</td></tr>
    // <tr><td> Microsoft SQL Server</td><td>
    // Prior to SQL Server 2005, schema support is similar to Oracle: "schema" is synonymous with 
    // "owner".  As of SQL Server 2005, schema is supported as a separate concept, and a user's 
    // default schema can be configured (though it still defaults to a schema with the same name
    // as the user).  It is not possible to set the default schema in the JDBC connection URL</td></tr>
    // <tr><td> MySQL</td><td>
    // MySQL does not have a separate concept of "schema"; it treats the terms "schema" and 
    // "database" interchangeably.  In fact MySQL databases actually behave more like schemas, in 
    // that a connection to database X can refer to a table in database Y simply by qualifying 
    // the name in the query.  Also, because schema and database are the same concept in MySQL, 
    // overriding the "default schema" is done implicitly when you specify which database to 
    // connect to in your JDBC connection URL</td></tr>
    // <tr><td> Oracle</td><td>
    // Arbitrarily named schemas are not supported; in Oracle, "schema" is synonymous with "user",
    // so each valid user in the database is associated implicitly with a schema of the same name,
    // and there are no other schemas possible.  It is possible to refer to tables in another
    // user's schema (assuming you have the privileges to do so) by simply qualifying the table name.
    // The default schema is always implied by the connecting user and cannot be overridden.</td></tr>
    // <tr><td> Postgres</td><td>
    // Arbitrarily named schemas are supported.
    // Rather than the concept of a "default schema", Postgres supports the idea of a search path 
    // of schemas, whereby unqualified table references cause a search of the list of schemas in 
    // order, and the first schema in the path is the "current" one for creation purposes.  
    // Unfortunately, there is no way to specify this search path on the JDBC connection URL, so
    // the default schema comes from the user definition, ultimately defaulting to the default 
    // "public" schema</td></tr>
    // </table>
    // @serverDS only
    // @visibility external
    //<
    

    //> @attr dataSource.dbName (String : null : [IR])
    // For DataSources using the +link{group:sqlDataSource,SmartClient SQL engine} for
    // persistence, which database configuration to use.  Database configurations can be
    // created using the +link{group:adminConsole, Admin Console}.  If unset, the default
    // database configuration is used (which is also settable using the "Databases" tab).
    // 
    // @group serverDataIntegration
    // @requiresModules SCServer
    // @serverDS only
    // @visibility external
    //<

    //> @attr dataSource.configBean (String : null : [IRA])
    // For DataSources of +link{serverType,serverType} "hibernate", the name of a Spring 
    // bean to query to obtain Hibernate Configuration for this particular DataSource.  Note
    // that this is intended for DataSource-specific configuration overrides for unusual 
    // circumstances, such as a DataSource whose physical data store is a completely 
    // different database to that used by other DataSources.  See the 
    // +link{group:hibernateIntegration,Integration with Hibernate} article for more 
    // information
    // 
    // @group serverDataIntegration
    // @requiresModules SCServer
    // @serverDS only
    // @visibility external
    //<

    //> @attr dataSource.quoteColumnNames (boolean : null : [IR])
    // If set, tells the SQL engine to quote column names in all generated DML and DDL 
    // statements for this dataSource.  This will ensure that queries generated against
    // tables that do not follow the database product's natural column-naming conventions 
    // will still work.
    // <p>
    // In general we recommend that you allow the database to use its natural naming scheme
    // when creating tables (put more simply, just do not quote column names in the 
    // <code>CREATE TABLE</code> statement); if you do this, you will not need to worry about 
    // quoting column names when querying.  However, if you are dealing with pre-existing 
    // tables, or do not have control over the database naming conventions used, this property
    // may become necessary.  This property may also be necessary if you are using field/column
    // names that clash with reserved words in the underlying database (these vary by database,
    // but a field called "date" or "timestamp" would have problems with most database products)
    // <p>
    // <b>Note:</b> Only applicable to dataSources of +link{attr:serverType,serverType} "sql".
    //
    // @serverDS only
    // @visibility external
    //<
    
    //> @attr dataSource.jsonPrefix (String : null : [IRA])
    // Allows you to specify an arbitrary prefix string to apply to all json format responses 
    // sent from the server to this application.
    // <P>
    // The inclusion of such a prefix ensures your code is not directly executable outside of
    // your application, as a preventative measure against
    // +externalLink{http://www.google.com/search?q=javascript+hijacking, javascript hijacking}.
    // <P>
    // Only applies to responses formatted as json objects. Does not apply to responses returned
    // via scriptInclude type transport.<br>
    // Note: If the prefix / suffix served by your backend is not a constant, you can use 
    // +link{operationBinding.dataFormat, dataFormat:"custom"} instead and explicitly parse
    // the prefix out as part of +link{dataSource.transformResponse(), transformResponse()}.
    //
    // @see operationBinding.dataFormat
    // @see operationBinding.dataTransport
    // @serverDS allowed
    // @visibility external
    //<
    
    //> @attr dataSource.jsonSuffix (String : null : [IRA])
    // Allows you to specify an arbitrary suffix string to apply to all json format responses 
    // sent from the server to this application.
    // <P>
    // The inclusion of such a suffix ensures your code is not directly executable outside of
    // your application, as a preventative measure against
    // +externalLink{http://www.google.com/search?q=javascript+hijacking, javascript hijacking}.
    // <P>
    // Only applies to responses formatted as json objects. Does not apply to responses returned
    // via scriptInclude type transport.
    //
    // @see operationBinding.dataFormat
    // @see operationBinding.dataTransport
    // @serverDS allowed
    // @visibility external
    //<

    //> @attr dataSource.fields         (Array of DataSourceField : null : [IR])
    // The list of fields that compose records from this DataSource.
    // <P>
    // Each DataSource field can have type, user-visible title, validators, and other metadata
    // attached.
    //
    // @group fields
    // @see class:DataSourceField
    // @serverDS allowed
    // @visibility external
    //<

    //> @attr dataSource.inheritsFrom (DataSource ID : null : IR)
    // ID of another DataSource this DataSource inherits its +link{fields} from.
    // <P>
    // Local fields (fields defined in this DataSource) are added to inherited fields 
    // to form the full set of fields.  Fields with the same name are merged in the same way
    // that +link{DataBoundComponent.fields,databound component fields} are merged with
    // DataSource fields.
    // <P>
    // The default order of the combined fields is new local fields first (including any fields
    // present in the parent DataSource which the local DataSource re-declares), then parent
    // fields.  You can set +link{useParentFieldOrder} to instead use the parent's field
    // order, with new local fields appearing last.  You can set +link{showLocalFieldsOnly} to
    // have all non-local fields hidden.
    // <P>
    // Note that <b>only fields are inherited</b> - other properties such as dataURL and
    // dataFormat are not.  You can use ordinary inheritance, that is, creating a subclass of
    // DataSource, in order to share properties such as dataURL across a series of DataSources
    // that also inherit fields from each other via <code>inheritsFrom</code>.
    // <P>
    // This feature can be used for:
    // <ul>
    // <li>creating a customized view (eg, only certain fields shown) which will be used by
    // multiple +link{DataBoundComponent,databound components}.
    // <li>adding presentation-specific attributes to metadata that has been automatically
    // derived from +link{XMLTools.loadXMLSchema,XML Schema} or other metadata formats
    // <li>modeling object subclassing and extension in server-side code and storage systems
    // <li>modeling relational database joins, and the equivalents in other systems
    // <li>creating hooks for others to customize your application in a maintainable way.  For
    // example, if you have a dataSource "employee", you can create a dataSource
    // "customizedEmployee" which inherits from "employee" but does not initially define any
    // fields, and bind all +link{DataBoundComponent,databound components} to
    // "customizedEmployee".  Customizations of fields (including appearance changes, field
    // order, new fields, hiding of fields, and custom validation rules) can be added to
    // "customizedEmployee", so that they are kept separately from the original field data and
    // have the best possible chance of working with future versions of the "employee"
    // dataSource.
    // </ul>
    //
    // @group fields
    // @serverDS allowed
    // @visibility external
    // @example schemaChaining
    //<

    
    //> @type DSInheritanceMode
    // For DataSources of type "sql", specifies the kind of inheritance to use when a dataSource
    // specifies +link{dataSource.inheritsFrom,inheritsFrom}.
    //
    // @value "full"
    // Inherit fields by copying them onto the inheriting DataSource's underlying table.  When we 
    // import a DataSource with this inheritanceMode, we create actual columns for inherited fields
    //  on the table we create.  With this inheritanceMode, inherited fields are updatable.
    //
    // @value "none"
    // Do not physically inherit fields onto the inheriting DataSource's SQL table.  Columns will
    // not be created for inherited fields on import, and all generated SQL operations will exclude
    // inherited fields.  However, those fields are still part of the DataSource's schema so you
    // can, for example, write +link{group:customQuerying,custom SQL} that returns values for 
    // the inherited fields, and they will be delivered to the client.
    // 
    // @group fields
    // @serverDS only
    // @visibility external
    //<
    
    
    
    
    //> @attr dataSource.inheritanceMode (DSInheritanceMode : "full" : IR)
    // For dataSources of +link{serverType,serverType} "sql" only, specifies the inheritance 
    // mode to use.  This property has no effect for any other type of DataSource.
    //
    // @see dataSource.inheritsFrom
    // @group fields
    // @serverDS only
    // @visibility external
    //<


    //> @attr dataSource.useFlatFields (Boolean : null : IR)
    // Like +link{dataBoundComponent.useFlatFields}, but applies to all DataBound components
    // that bind to this DataSource.
    //
    // @group fields
    // @serverDS allowed
    // @visibility external
    //<

    //> @attr dataSource.showLocalFieldsOnly (boolean : null : IR)
    // For a DataSource that inherits +link{fields} from another DataSource 
    // (via +link{inheritsFrom}), indicates that only the fields listed in this DataSource
    // should be shown.  All other inherited parent fields will be marked "hidden:true".
    //
    // @group fields
    // @serverDS allowed
    // @visibility external
    //<

    //> @attr dataSource.useParentFieldOrder (boolean : null : IR)
    // For a DataSource that inherits +link{fields} from another DataSource 
    // (via +link{inheritsFrom}), indicates that the parent's field order should be used
    // instead of the order of the fields as declared in this DataSource.  New fields, if any,
    // are placed at the end.
    // 
    // @group fields
    // @serverDS allowed
    // @visibility external
    // @example schemaChaining
    //<

    //> @attr dataSource.dropExtraFields     (boolean : null : IR)
    // Indicates that for server responses, for any data being interpreted as DataSource records, 
    // only data that
    // corresponds to declared fields should be retained; any extra fields should be discarded.
    // <P>
    // For +link{DataSource.dataFormat,JSON} data, this means extra properties in selected
    // objects are dropped.
    // <P>
    // By default, for DMI DSResponses, DSResponse.data is filtered on the server to just
    // the set of fields defined on the DataSource.  This type of filtering can also be enabled
    // for non-DMI DSResponses (see the overview in +link{dmiOverview,DMI}).  Setting this property to
    // <code>false</code> disables this filtering for this DataSource only.  This setting
    // overrides the configuration in +link{server_properties,server.properties}.  This setting can
    // be overridden by +link{ServerObject.dropExtraFields}.
    // 
    // @group clientDataIntegration
    // @serverDS allowed
    // @visibility external
    //<

    //> @attr dataSource.sendExtraFields     (Boolean : true : IR)
    // Analogous to +link{dataSource.dropExtraFields}, for data sent to the server.  Setting this
    // attribute to false ensures that for any records in the data object, only fields that
    // correspond to declared dataSource fields will be present on the dsRequest data object passed
    // to +link{dataSource.transformRequest()} and ultimately sent to the server.
    // 
    // @group clientDataIntegration
    // @serverDS allowed
    // @visibility external
    //<
    sendExtraFields:true,
    
    //> @attr dataSource.autoDeriveSchema (boolean : null : IR)
    // This property allows you to specify that your DataSource's schema (field definitions) should
    // be automatically derived from some kind of metadata.  This causes SmartClient to create 
    // a special super-DataSource, which is used purely as a source of default schema for this 
    // DataSource; this is arranged by causing the autoDerived DataSource to automatically 
    // +link{inheritsFrom,inherit from} the special super-DataSource.  This allows you to 
    // auto-derive schema from existing metadata, whilst still being able to override any or all
    // of it as required.<p>
    // This property has a different implementation depending upon the 
    // +link{dataSource.serverType,serverType} of the DataSource:<ul>
    // <li>For a DataSource with serverType: "sql", automatically derive the dataSource's schema 
    // from the Spring bean or Java class specified in +link{schemaBean,schemaBean}.  If 
    // <code>schemaBean</code> is not specified, derive the schema from the columns in the SQL 
    // table specified in +link{tableName,tableName}.  More information on SQL DataSources 
    // is +link{group:sqlDataSource,here}</li>  
    // <li>For a DataSource with serverType: "hibernate", automatically derive the dataSource's
    // schema from the Spring bean, Hibernate mapping or Java class named in the 
    // +link{schemaBean,schemaBean} property.  If no such thing exists, derive the schema from
    // the Hibernate mapping or Java class specified in the +link{beanClassName,beanClassName}
    // property (this allows you to specify schema and mapping separately, in the unusual 
    // circumstance that you have a need to do so).  Note that the "mappings" referred to here
    // can mean either <code>.hbm.xml</code> files or annotated classes; both are supported.
    // If neither of these is successful, derive the schema from the underlying SQL table 
    // specified in +link{tableName,tableName}.  More information on Hibernate DataSources is
    // +link{group:hibernateIntegration,here}</li>  
    // <li>For a DataSource with serverType: "jpa1" or "jpa", automatically derive the 
    // dataSource's schema from the annotated JPA class named in the +link{schemaBean,schemaBean} 
    // property.  If the schemaBean property is not defined, derive the schema from the 
    // annotated JPA class named in the +link{beanClassName,beanClassName} property (as with 
    // Hibernate, this allows you to specify schema and mapping separately if you need to do 
    // so).  JPA DataSource generation relies on annotations (the orm.xml mapping file is not 
    // supported).  More information on JPA DataSources is +link{group:jpaIntegration,here}</li>  
    // <li>For other DataSource types, attempt to find a Spring bean with the name specified in
    // the +link{schemaBean,schemaBean} property.  If no such bean is found (or Spring is not 
    // present), attempt to instantiate an object whose fully-qualified class name is the value
    // in the <code>schemaBean</code> property.  If one of these approaches succeeds, we derive
    // the schema from the discovered object (by treating it as a Java Bean and assuming that 
    // each one of its getters corresponds to a like-named field in the DataSource).  More 
    // information on custom DataSource implementations is +link{group:writeCustomDataSource,here}.</li>
    // </ul>
    // The following table shows how SQL types are derived into 
    // +link{type:FieldType,DataSource types} when we use an SQL table as a source of metadata
    // for a SQL or Hibernate DataSource:
    // <table border="1" class="normal">
    // <tr><th>SQL type</th><th>+link{attr:DataSourceField.type,DataSource type}</th></tr>
    // <tr><td>CHAR, VARCHAR, LONGVARCHAR, TEXT, CLOB</td><td>text</td></tr>
    // <tr><td>BIT, TINYINT, SMALLINT, INTEGER, BIGINT, DECIMAL<sup>*</sup>, NUMBER<sup>**</sup></td><td>integer</td></tr>
    // <tr><td>REAL, FLOAT, DOUBLE, DECIMAL<sup>*</sup>, NUMBER<sup>**</sup></td><td>float</td></tr>
    // <tr><td>DATE</td><td>date</td></tr>
    // <tr><td>TIME</td><td>time</td></tr>
    // <tr><td>TIMESTAMP</td><td>datetime</td></tr>
    // <tr><td>BLOB, BINARY, VARBINARY, LONGVARBINARY</td><td>binary</td></tr>
    // </table>
    // <sup>*</sup>For DECIMAL types, we inspect the "DECIMAL_DIGITS" attribute of the JDBC 
    // metadata and designate the field type "integer" if that attribute is 0, or "float" if
    // it is some other value.<br>
    // <sup>**</sup>NUMBER is an Oracle-only type that appears in the JDBC metadata as DECIMAL
    // and is handled exactly the same way as DECIMAL
    // <p>
    // The following table shows how Java types are derived into DataSource types when we use
    // an unannotated Java class (Spring bean, Hibernate mapping or POJO) as a source of 
    // metadata for a SQL, Hibernate or custom DataSource:
    // <table border="1" class="normal">
    // <tr><th>Java type</th><th>+link{attr:DataSourceField.type,DataSource type}</th></tr>
    // <tr><td>boolean, Boolean</td><td>boolean</td></tr>
    // <tr><td>char, Character, String, Reader</td><td>text</td></tr>
    // <tr><td>byte, short, int, long, Byte, Short, Integer, Long, BigInteger</td><td>integer</td></tr>
    // <tr><td>float, double, Float, Double, BigDecimal</td><td>float</td></tr>
    // <tr><td>Enum</td><td>enum (see discussion below)</td></tr>
    // <tr><td>InputStream</td><td>binary</td></tr>
    // <tr><td>java.sql.Date, java.util.Date, java.util.Calendar</td><td>date</td></tr>
    // <tr><td>java.sql.Time</td><td>time</td></tr>
    // <tr><td>java.sql.Timestamp</td><td>datetime</td></tr>
    // </table>
    // <p>
    // We also derive the following information for Hibernate mappings described in 
    // <code>.hbm.xml</code> files (for Hibernate mappings described in Java annotations, see
    // the next table):
    // <ul>
    // <li>The first field we encounter that is described in the mapping with an &lt;id&gt; tag
    //     is marked as a primaryKey</li>
    // <li>If that field is marked as being generated, we set its type to "sequence"</li>
    // </ul>
    // Finally, this table shows how Java types are derived into DataSource types when we use an 
    // annotated class as a source of metadata.  Note annotated classes are necessary for JPA
    // DataSources, but you can choose to use them for other types of DataSource as well.  
    // For Hibernate DataSources, this is very worthwhile because Hibernate will also make use
    // of the annotations as config, meaning you don't need to specify <code>.hbm.xml</code> 
    // files.  For SQL and custom DataSources, there is no benefit at the persistence level, 
    // but it may still be worthwhile because the use of an annotated class gives us better 
    // metadata and allows us to generate a better, closer-fitting autoDerive DataSource than 
    // we can from examination of SQL schema or plain Java Beans:
    // <table border="1" class="normal">
    // <tr><th>Java type</th><th>+link{attr:DataSourceField.type,DataSource type}</th></tr>
    // <tr><td>boolean, Boolean</td><td>boolean</td></tr>
    // <tr><td>char, Character, String, Reader</td><td>text</td></tr>
    // <tr><td>byte, short, int, long, Byte, Short, Integer, Long, BigInteger</td><td>integer</td></tr>
    // <tr><td>float, double, Float, Double, BigDecimal</td><td>float</td></tr>
    // <tr><td>InputStream</td><td>binary</td></tr>
    // <tr><td>java.util.Date (with Temporal set to DATE), java.sql.Date</td><td>date</td></tr>
    // <tr><td>java.util.Date (with Temporal set to TIME), java.sql.Time</td><td>time</td></tr>
    // <tr><td>java.util.Date (with Temporal set to TIMESTAMP), java.util.Calendar, java.sql.Timestamp</td><td>datetime</td></tr>
    // <tr><td>Enum (with Enumerated set to STRING)</td><td>enum (see discussion below)</td></tr>
    // <tr><td>Enum (with Enumerated set to ORDINAL)</td><td>intEnum (see discussion below)</td></tr>
    // <tr><td>Field with Lob annotation</td><td>binary</td></tr>
    // <tr><td>Field with GeneratedValue annotation</td><td>sequence</td></tr>
    // </table>
    // <p><p>
    // <h4>enums and valueMaps</h4>
    // When we come across Java <code>Enum</code> properties in plain or annotated classes, 
    // as well as setting the field type as noted in the above tables, we also generate a 
    // valueMap for the field, based on the <code>Enum</code> members.  
    // <p>
    // For cases where we generate a field of SmartClient type "enum" (see the above tables), 
    // the keys of the valueMap are the result of calling <code>name()</code> on each member
    // of the underlying Java Enum (in other words, its value exactly as declared in its 
    // enum declaration).  For cases where we generate a field of SmartClient type "intEnum",
    // the keys of the valueMap are strings representing the ordinal number of each member 
    // in the Java Enum - "0", "1", etc. Note that this behavior will be overriden by 
    // +link{attr:DataSource.enumTranslateStrategy,DataSource.enumTranslateStrategy} if both are set.
    // <p>
    // In both of these case, the display values generated for the valueMap are the result 
    // of calling <code>toString()</code> on each Enum member.  If that gives the same 
    // value as calling <code>name()</code>, the value is passed through 
    // <code>DataTools.deriveTitleFromName()</code>, which applies the same processing rules
    // as +link{classMethod:DataSource.getAutoTitle()} to derive a more user-friendly display value.
    // <p>
    // <b>NOTE:</b> <code>schemaBean</code> implies <code>autoDeriveSchema</code>, because it 
    // has no other purpose than to name the bean to use for auto-derived schema.  Thus, if you 
    // specify <code>schemaBean</code> you do not need to specify <code>autoDeriveSchema</code> 
    // as well (though it does no harm to do so).  However, <code>tableName</code> can be 
    // validly specified without implying <code>autoDeriveSchema</code>, so in that case you 
    // must explicitly specify <code>autoDeriveSchema</code>.
    // <p>
    // <b>NOTE:</b> The underlying super-DataSource is cached in server memory, so that it does 
    // not have to be derived and created each time you need it.  However, the cache manager 
    // will automatically refresh the cached copy if it detects that the deriving DataSource 
    // has changed.  Thus, if you change the metadata your DataSource is deriving (if, for 
    // example, you add a column to a table), all you need to do is touch the 
    // <code>.ds.xml</code> file (ie, update its last changed timestamp - you don't actually
    // have to change it) and the cached copy will be refreshed next time it is needed.
    // <p>
    // When autoDeriveSchema is set, SQLDataSource will automatically discover foreignKeys and
    // deliver table and column name information to the client in hashed form so that two
    // DataSources that are linked by native SQL foreign keys will automatically discover each
    // other if loaded into the same application, and set dataSourceField.foreignKey
    // automatically.  Because the table and column names are delivered as cryptohashes, there
    // is no information leakage, but regardless the feature can be disabled via setting
    // datasource.autoLinkFKs to false in +link{server_properties,server.properties}.  
    // This hashed linkage information is
    // delivered to the client in properties +link{dataSource.tableCode} 
    // and +link{dataSourceField.fkTableCode}/+link{dataSourceField.fkColumnCode,fkColumnCode}
    // 
    // @group fields
    // @serverDS only
    // @visibility external
    //<

    //> @attr dataSource.schemaBean (string : null : IR)
    // For DataSources that specify +link{autoDeriveSchema}, this property indicates the name
    // of the Spring bean, Hibernate mapping or fully-qualified Java class to use as parent 
    // schema.
    //
    // @see autoDeriveSchema
    // @group fields
    // @serverDS only
    // @visibility external
    //<

    //> @attr dataSource.idClassName (string : null : IR)
    // For JPA and Hibernate DataSources this property indicates, that data source has composite primary key and
    // specifies fully-qualified Java class:<ul>
    // <li>with <b><code>@EmbeddedId</code></b> you have to specify class name of declared id</li>
    // <li>with <b><code>@IdClass</code></b> you have to specify class specified in annotation declaration</li></ul>
    //
    // @group fields
    // @serverDS only
    // @visibility external
    //<

    // XML
    // ---------------------------------------------------------------------------------------
    
    //> @attr dataSource.xmlNamespaces (Object : See below : IR)
    // Sets the XML namespace prefixes available for XPaths on a DataSource-wide basied.  See
    // +link{operationBinding.xmlNamespaces} for details.
    //
    // @group clientDataIntegration
    // @serverDS allowed
    // @visibility xmlBinding
    //<

    //> @attr dataSource.serviceNamespace (URN : null : IR)
    // For an XML DataSource, URN of the WebService to use to invoke operations.  This URN
    // comes from the "targetNamespace" attribute of the &lt;wsdl:definitions&gt; element in a
    // WSDL (Web Service Description Language) document, and serves as the unique identifier of
    // the service.
    // <P>
    // Having loaded a WebService using +link{XMLTools.loadWSDL()}, setting
    // <code>serviceNamespace</code> combined with specifying 
    // +link{class:OperationBinding,operationBindings}
    // that set +link{attr:operationBinding.wsOperation} will cause a DataSource to invoke web
    // service operations to fulfill DataSource requests (+link{class:DSRequest,DSRequests}).
    // <P>
    // Setting <code>serviceNamespace</code> also defaults
    // +link{DataSource.dataURL,dataURL} to the service's location,
    // +link{DataSource.dataFormat,dataFormat} to "xml" and
    // +link{operationBinding.dataProtocol,dataProtocol} to "soap".
    //
    // @group wsdlBinding
    // @group clientDataIntegration
    // @serverDS allowed
    // @visibility xmlBinding
    // @example wsdlBinding
    //<

    //> @attr dataSource.schemaNamespace (URN : null : R)
    // For a DataSource derived from WSDL or XML schema, the XML namespace this schema belongs
    // to.  This is a read-only attribute automatically present on DataSources returned from
    // +link{schemaSet.getSchema()} and +link{webService.getSchema()}.
    //
    // @group wsdlBinding
    // @group clientDataIntegration
    // @serverDS allowed
    // @visibility xmlBinding
    //<

    //> @attr dataSource.recordXPath          (XPathExpression : null : [IR])
    // See +link{attr:operationBinding.recordXPath}.  <code>recordXPath</code> can be specified
    // directly on the DataSource for a simple read-only DataSource only capable of "fetch"
    // operations.
    //
    // @group clientDataIntegration
    // @serverDS allowed
    // @visibility xmlBinding
    // @example xmlDataSource
    // @example jsonXPath
    //<

    //> @attr dataSource.dataURL              (URL : null : [IR])
    // Default URL to contact to fulfill all DSRequests.  Can also be set on a
    // per-operationType basis via +link{attr:operationBinding.dataURL}.
    // <P>
    // NOTE: Best practice is to use the same <code>dataURL</code> for all DataSources which
    // fulfill DSRequests via the server-side RPCManager API.  Otherwise, cross-DataSource
    // +link{RPCManager.startQueue(),operation queuing} will not be possible.
    //
    // @group clientDataIntegration
    // @serverDS allowed
    // @visibility xmlBinding
    // @example jsonDataSource
    //<

    //> @attr dataSource.tagName             (String : null : IRA)
    // Tag name to use when serializing to XML.  If unspecified, the <code>dataSource.ID</code>
    // will be used.
    // @group clientDataIntegration
    // @serverDS allowed
    // @visibility xmlBinding
    //<

    
    //> @attr dataSource.useOfflineStorage (boolean : null : IRW)
    // Whether we store server responses for this DataSource into 
    // +link{class:Offline,browser-based offline storage}, and then use those stored responses
    // at a later time if we are offline (ie, the application cannot connect to the server).  
    // Note that by default we do NOT use offline storage for a dataSource.
    // @serverDS allowed
    // @group offlineGroup
    // @visibility external
    //<



// whether to transform XML responses to JS automatically, such that the client-side DSResponse
// is a JS object (whether the transform was performed on client or server is intended to be an
// implementation detail).
// At the moment 2005.7.5, some components can work directly with XML objects (eg read-only,
// selectable ListGrid), some can't.  Server XML->JS transform will always speed up the client.
// Aside from features that don't yet work with XML object, client XML->JS transform is a
// tradeoff between initial load vs later access time.
transformResponseToJS:true,

// whether this DataSource can queue requests to be submitted as a batch
supportsRequestQueuing : true,

    // Client Only DataSources
	// ----------------------------------------------------------------------------------------
    //> @attr dataSource.clientOnly     (Boolean : false : [IR])
    // A clientOnly DataSource simulates the behavior of a remote data store by manipulating a
    // static dataset in memory as +link{DSRequest,DSRequests} are executed on it.  Any changes
    // are lost when the user reloads the page or navigates away.
    // <P>
    // A clientOnly DataSource will return responses asynchronously, just as a DataSource
    // accessing remote data does.  This allows a clientOnly DataSource to be used as a
    // temporary placeholder while a real DataSource is being implemented - if a clientOnly
    // DataSource is replaced by a DataSource that accesses a remote data store, UI code for
    // components that used the clientOnly DataSource will not need be changed.
    // <P>
    // A clientOnly DataSource can also be used as a shared cache of modifiable data across
    // multiple UI components when immediate saving is not desirable.  In this case, several
    // components may interact with a clientOnly DataSource and get the benefit of
    // +link{ResultSet} behaviors such as automatic cache sync and in-browser data filtering
    // and sorting.  When it's finally time to save, +link{dataSource.cacheData} can be
    // inspected for changes and data can be saved to the original DataSource via
    // +link{addData()}, +link{updateData()} and +link{removeData()}, possibly in a
    // +link{rpcManager.startQueue,transactional queue}.  Note that
    // +link{DataSource.getClientOnlyDataSource()} lets you easily obtain a
    // <code>clientOnly</code> DataSource representing a subset of the data available from a
    // normal DataSource.  
    // <P>
    // See also +link{cacheAllData} - a <code>cacheAllData</code> behaves like a write-through
    // cache, performing fetch and filter operations locally while still performing remote save
    // operations immediately.
    // <P>
    // ClientOnly DataSources can be populated programmatically via +link{cacheData} - see
    // +link{group:clientOnlyDataSources,this discussion} for other ways to populate a
    // client-only DataSource with data.
    //
    // @group clientOnlyDataSources
    // @serverDS allowed
    // @visibility external
    // @example localDataSource
    //<
	//clientOnly: false,

    // in clientOnly mode, whether to shallow copy results to more fully simulate a server.
    // This allows separate changes to be made to the "server data set" (testData) and
    // ResultSet caches.
    copyLocalResults:true,
    // Also support a mode where we explicitly clone the local-data. This will recursively
    // clone sub-objects. Off by default - not usually necessary and a little risky as
    // data may have references to non-serializable objects.
    deepCopyLocalResults:false,

    // Filtering
	// ----------------------------------------------------------------------------------------
	// whether to match case in filter    
	//filterIsCaseSensitive : false,		

    //> @attr dataSource.criteriaPolicy (CriteriaPolicy : "dropOnShortening" : IRWA)
    // Decides under what conditions the +link{ResultSet} cache should be dropped when the
    // +link{resultSet.criteria} changes.
    // @see compareCriteria()
    // @serverDS allowed
    // @visibility external
    //<
    criteriaPolicy:"dropOnShortening",

    // ResultTrees
	// ----------------------------------------------------------------------------------------
    //> @attr dataSource.resultTreeClass (Class Object : null : [IRA])
    // Class for ResultTrees used by this datasource.  If null, defaults to using
    // +link{ResultTree}.
    // <P>
    // This can be set to a custom subclass of ResultTree that, for example, hangs on to extra
    // information necessary for integration with web services.
    //
    // @serverDS allowed
    //  @visibility external
    //<

    // ResultSets
	// ----------------------------------------------------------------------------------------
    //> @attr dataSource.resultSetClass (Class Object : null : [IRA])
    // Class for ResultSets used by this datasource.  If null, defaults to using
    // +link{ResultSet}.
    // <P>
    // This can be set to a custom subclass of ResultSet that, for example, hangs onto to extra
    // information necessary for integration with web services.
    //
    // @serverDS allowed
    // @visibility external
    //<
    
    // Validation
	// ----------------------------------------------------------------------------------------
    //> @attr dataSource.useLocalValidators (boolean : null : IRWA)
    // Whether to attempt validation on the client at all for this DataSource.  If unset (the
    // default), client-side validation is enabled.
    // <p>
    // Disabling client-side validation entirely is a good way to test server-side validation.
    //
    // @group validation
    // @serverDS allowed
    // @visibility external
    //<
    // NOTE: code that checks this property interprets null as true

    //> @attr dataSource.requiredMessage (string : null : [IRW])
    // The required message when a field that has been marked as
    // +link{DataSourceField.required,required} is not filled in by the user.
    // @group formTitles
    // @visibility external
    //<
    
    // ShowPrompt
	// ----------------------------------------------------------------------------------------
    //> @attr dataSource.showPrompt (Boolean : true : IRW)
    // Whether RPCRequests sent by this DataSource should enable
    // +link{attr:RPCRequest.showPrompt} in order to block user interactions until the
    // request completes.  
    // <p>
    // DataSource requests default to blocking UI interaction because, very often, if the user
    // continues to interact with an application that is waiting for a server response, some
    // kind of invalid or ambiguous situation can arise.
    // <p>
    // Examples include pressing a "Save" button a second time before the first save completes,
    // making further edits while edits are still being saved, or trying to initiate editing on
    // a grid that hasn't loaded data.
    // <p>
    // Defaulting to blocking the UI prevents these bad interactions, or alternatively, avoids
    // the developer having to write repetitive code to block invalid interactions on every
    // screen.
    // <p>
    // If an operation should ever be non-blocking, methods that initiate DataSource requests
    // (such as +link{DataSource.fetchData()}) will generally have a
    // <code>requestProperties</code> argument allowing <code>showPrompt</code> to be set to
    // false for a specific request.
    //
    // @serverDS allowed
    // @visibility external
    //<
    showPrompt:true,

    // Selection
	// ----------------------------------------------------------------------------------------
    //> @attr dataSource.selectionClass (className : null : [A])
    // Class to use as the Selection object for ResultSets derived from this DataSource.
    // @visibility serverSelection
    //<
    

    // Java enum translation
	// ----------------------------------------------------------------------------------------
    //> @type EnumTranslateStrategy
    // Determines how Java enums are translated to and from Javascript by the SmartClient server.
    //
    // @value "name"
    //   Translates to/from a String matching the constant name. This is the default if not set.
    // @value "string"
    //   Translates to/from a String matching the <code>enum.toString()</code>.
    // @value "ordinal"
    //   Translates to/from an integer matching the ordinal number of the constant within 
    //   the enumeration
    // @value "bean"
    //   Translates to/from a Javascript object containing one property for each property defined 
    //   within the enum. The constant itself and the ordinal number are included in the JS object.
    //   By default they are called "_constant" and "_ordinal", but this can be overridden with 
    //   the +link{dataSource.enumOrdinalProperty} and +link{dataSource.enumConstantProperty}
    //   properties
    //
    // @see dataSource.enumTranslateStrategy
    // @serverDS allowed
    // @visibility external
    //<
    
    
    //> @attr dataSource.enumTranslateStrategy (EnumTranslateStrategy : null : IA)
    //
    // Sets the strategy this DataSource uses to translate Java enumerated types (objects of type
    // enum) to and from Javascript.
    // 
    // This property is only applicable if you are using the SmartClient server
    // @serverDS allowed
    // @visibility external
    //<

    //> @attr dataSource.enumConstantProperty (String : null : IA)
    //
    // The name of the property this DataSource uses for constant name when translating
    // Java enumerated types to and from Javascript, if the +link{enumTranslateStrategy} is set
    // to "bean".  Defaults to "_constant" if not set.
    // <p>
    //  This property is only applicable if you are using the SmartClient server
    // @serverDS allowed
    // @visibility external
    //<

    //> @attr dataSource.enumOrdinalProperty (String : null : IA)
    //
    // The name of the property this DataSource uses for ordinal number when translating
    // Java enumerated types to and from Javascript, if the +link{enumTranslateStrategy} is set
    // to "bean".  Defaults to "_ordinal" if not set.
    // <p>
    //  This property is only applicable if you are using the SmartClient server
    // @serverDS allowed
    // @visibility external
    //<

    //> @attr dataSource.autoDeriveTitles (boolean : true : IR)
    // If set, titles are automatically derived from +link{dataSourceField.name,field.name} for any 
    // field that does not have a +link{dataSourceField.title,field.title} and is not marked
    // +link{dataSourceField.hidden,hidden}:true, by calling the method +link{getAutoTitle()}.
    // @serverDS allowed
    // @visibility external
    //<
    autoDeriveTitles: true,

    // Multi-level Sorting
    //> @attr dataSource.canMultiSort (boolean : true : IR)
    // When true, indicates that this DataSource supports multi-level sorting.
    // @serverDS allowed
    // @visibility external
    //<
    canMultiSort: true,
    
    //> @attr dataSource.sparseUpdates (boolean : false : IR)
    // When true, indicates that any updates for this DataSource include only those fields 
    // that have actually changed (and primaryKey fields); when false (the default), all 
    // field values are included in updates, whether they have changed or not
    // @serverDS only
    // @visibility external
    //<
    
    //> @attr dataSource.noNullUpdates (boolean : false : IR)
    // When true, indicates that fields in this DataSource will never be positively updated
    // to the null value; they may arrive at null values by being omitted, but we will 
    // not send actual null values in update requests.  When false (the default), null is 
    // not treated in any special way.
    // <p>
    // Setting this value causes null-assigned fields to be replaced with the field's
    // +link{DataSourceField.nullReplacementValue,nullReplacementValue}, if one is declared.
    // If no <code>nullReplacementValue</code> is declared for the field, the null assignment
    // is replaced with the DataSource's +link{nullStringValue,nullStringValue}, 
    // +link{nullIntegerValue,nullIntegerValue}, +link{nullFloatValue,nullFloatValue} 
    // or +link{nullDateValue,nullDateValue}, depending on the field type.
    // <p>
    // For "add" operations, setting +link{omitNullDefaultsOnAdd,omitNullDefaultsOnAdd} causes
    // null-valued fields to be removed from the request entirely, rather than replaced with
    // default values as described above.
    // @serverDS only
    // @visibility external
    //<
    
    //> @attr dataSource.omitNullDefaultsOnAdd (boolean : false : IR)
    // When true, and +link{noNullUpdates,noNullUpdates} is also true, indicates that "add" 
    // requests on this DataSource will have null-valued fields removed from the request 
    // entirely before it is sent to the server, as opposed to the default behavior of 
    // replacing such null values with defaults.
    // @see noNullUpdates,noNullUpdates
    // @serverDS only
    // @visibility external
    //<
    
    //> @attr dataSource.nullStringValue (String : "" : IR)
    // If +link{noNullUpdates} is set, the value to use for any text field that has a null
    // value assigned on an update operation, and does not specify an explicit
    // +link{DataSourceField.nullReplacementValue,nullReplacementValue}.
    // @see noNullUpdates,noNullUpdates
    // @see DataSourceField.nullReplacementValue
    // @serverDS only
    // @visibility external
    //<
    nullStringValue: "",
    
    //> @attr dataSource.nullIntegerValue (int : 0 : IR)
    // If +link{noNullUpdates} is set, the value to use for any integer field that has a null
    // value assigned on an update operation, and does not specify an explicit
    // +link{DataSourceField.nullReplacementValue,nullReplacementValue}.
    // @see noNullUpdates,noNullUpdates
    // @see DataSourceField.nullReplacementValue
    // @serverDS only
    // @visibility external
    //<
    nullIntegerValue: 0,
    
    //> @attr dataSource.nullFloatValue (float : 0.0 : IR)
    // If +link{noNullUpdates} is set, the value to use for any float field that has a null
    // value assigned on an update operation, and does not specify an explicit
    // +link{DataSourceField.nullReplacementValue,nullReplacementValue}.
    // @see noNullUpdates,noNullUpdates
    // @see DataSourceField.nullReplacementValue
    // @serverDS only
    // @visibility external
    //<
    nullFloatValue: 0.0,
    
    //> @attr dataSource.nullBooleanValue (boolean : false : IR)
    // If +link{noNullUpdates} is set, the value to use for any boolean field that has a null
    // value assigned on an update operation, and does not specify an explicit
    // +link{DataSourceField.nullReplacementValue,nullReplacementValue}.
    // @see noNullUpdates,noNullUpdates
    // @see DataSourceField.nullReplacementValue
    // @serverDS only
    // @visibility external
    //<
    nullBooleanValue: false,
    
    //> @attr dataSource.nullDateValue (Date : See below : IR)
    // If +link{noNullUpdates} is set, the value to use for any date or time field that has a
    // null value assigned on an update operation, and does not specify an explicit
    // +link{DataSourceField.nullReplacementValue,nullReplacementValue}.
    // <p>
    // Unlike strings and numbers, there is no "natural" choice for a null replacement value 
    // for dates.  The default value we have chosen is midnight on January 1st 1970, simply 
    // because this is "the epoch" - the value that is returned by calling "new Date(0)"
    // @see noNullUpdates,noNullUpdates
    // @see DataSourceField.nullReplacementValue
    // @serverDS only
    // @visibility external
    //<
    nullDateValue: new Date(0),
    

    // Role-based security
    // ----------------------------------------------------------------------------------------

    //> @attr dataSource.requiresAuthentication (boolean : null : IR)
    // Whether a user must be authenticated in order to access this DataSource.  This establishes a
    // default for the DataSource as a whole; individual +link{operationBindings} within the
    // DataSource may still override this setting by explicitly setting
    // +link{operationBinding.requiresAuthentication}.
    // <P>
    // Whether the user is authenticated is determined by calling
    // <code>httpServletRequest.getRemoteUser()</code>, hence works with both simple J2EE security
    // (realms and form-based authentication) and JAAS (Java Authentication & Authorization
    // Service).
    // <P>
    // If you wish to use an authentication scheme that does not make use of the servlet API's 
    // standards, SmartClient Server also implements the <code>setAuthenticated</code> method
    // on <code>RPCManager</code>.  You can use this API to tell SmartClient that all the 
    // requests in the queue currently being processed are associated with an authenticated
    // user; in this case, SmartClient will not attempt to authenticate the user via 
    // <code>httpServletRequest.getRemoteUser()</code>
    // <P>
    // You can set the default value for this property via setting "authentication.defaultRequired"
    // in +link{server_properties,server.properties}.  This allows you to, for example,
    // cause all DataSources to require
    // authentication for all operations by default.
    // <P>
    // Note that setting this property does not automatically cause an authentication mechanism to
    // appear - you still need to separately configure an authentication system.  Likewise, setting
    // requiresAuthentication="false" does not automatically allow users to bypass your authentication
    // mechanism - you need to set up a URL that will accept DSRequests and process them similar to
    // the default "IDACall" servlet, and which is not protected by the authentication system.  See
    // +link{group:servletDetails,Deploying SmartClient} for details on the IDACall servlet.
    //
    // @requiresModules SCServer
    // @group auth
    // @serverDS only
    // @visibility external
    //<

    //> @attr dataSource.requiresRole (String : null : IR)
    // Similar to +link{operationBinding.requiresRole}, but controls access to the DataSource as a
    // whole.
    //
    // @requiresModules SCServer
    // @group auth
    // @serverDS only
    // @visibility external
    //<

    //> @attr dataSource.requires (VelocityExpression : null : IR)
    // Indicates that the specified +link{VelocityExpression} must evaluate to true for a user 
    // to access this DataSource.
    // <P>
    // See also +link{operationBinding.requires}.
    //
    // @requiresModules SCServer
    // @group auth
    // @serverDS only
    // @visibility external
    //<

    //> @attr dataSource.creatorOverrides (boolean : null : IR)
    // Indicates that declarative security rules are waived for rows that were created by the 
    // current user.  Practically, this means that when a security check fails, instead of a 
    // security exception being thrown, we alter the criteria to ensure that the request can 
    // only return or affect rows that were created by the current authenticated user.  This 
    // allows you to create security regimes whereby users can see and edit data they created, 
    // but have access to other users' data forbidden or limited.
    // <p>
    // In order for this to work, we require two things:<ul>
    // <li>The DataSource must specify a field of type "creator" - this field type is described 
    //     on +link{type:FieldType,this page}</li>
    // <li>The authentication regime in use must include the idea of a "current user".  The 
    //     authentication provided by the Servlet API is an example of such a regime.</li>
    // </ul>
    // This setting can be overridden at operationBinding and field level, allowing extremely
    // fine-grained control.
    //
    // @requiresModules SCServer
    // @see operationBinding.creatorOverrides
    // @see dataSourceField.creatorOverrides
    // @group fieldLevelAuth
    // @serverDS only
    // @visibility external
    //<


    // Transactions
    // ----------------------------------------------------------------------------------------

    //> @attr dataSource.autoJoinTransactions (boolean : null : IR)
    // If true, causes all operations on this DataSource to automatically start or join a 
    // transaction; this means that multiple operations sent to the server in a 
    // +link{RPCManager.startQueue(),request queue} will be committed in a single transaction.
    // Note that this includes fetch operations - setting this property to true has the same
    // effect as a transaction policy of ALL for just this DataSource's operations - see the 
    // server-side <code>RPCManager.setTransactionPolicy()</code> for details of the different 
    // TransactionPolicy settings.
    // <p>
    // If this property is explicitly false, this causes all operations on this DataSource 
    // to be committed individually - the same as a transaction policy of NONE, just for this
    // DataSource's operations.
    // <p>
    // In either case, you can override the setting for individual operations - see 
    // +link{OperationBinding.autoJoinTransactions}.
    // <P>
    // If this property if null or not defined, we fall back to the default setting for this
    // type of DataSource.  These are defined in +link{server_properties,server.properties}
    // as follows:
    // <ul>
    // <li><b>Hibernate:</b><code> hibernate.autoJoinTransactions</code></li>
    // <li><b>JPA/JPA2:</b><code> jpa.autoJoinTransactions</code></li>
    // <li><b>SQL:</b> There is one setting per configured database connection (+link{dbName,dbName}).
    // For example, the setting for the default MySQL connection is 
    // <code> sql.Mysql.autoJoinTransactions</code></li>
    // </ul>
    // If the setting is not defined at the DataSource-type level, we use the system global 
    // default, which is defined in <code>server.properties</code> as 
    // <code>autoJoinTransactions</code>.
    // <P>
    // At the dbName and global system levels, you can set the autoJoinTransactions attribute 
    // to a valid Transaction Policy, rather than a simple true or false (although these 
    // values work too - true is the same as ALL, false is the same as NONE).  For valid 
    // TransactionPolicy values and their meanings, see the server-side Javadoc for 
    // <code>RPCManager.setTransactionPolicy()</code>
    // <P>
    // Note that the configuration settings discussed here can be overridden for a particular 
    // queue of requests by setting the server-side RPCManager's transaction policy.  Look in
    // the server-side Javadoc for <code>RPCManager.getTransactionPolicy()</code>.
    // <P>
    // NOTE: Setting this property to true does not cause a transactional persistence 
    // mechanism to automatically appear - you have to ensure that your DataSource supports 
    // transactions.  The SmartClient built-in SQL, Hibernate and JPA DataSources support transactions, 
    // but note that they do so <b>at the provider level</b>.  This means that you can combine 
    // updates to, say, an Oracle database and a MySQL database in the same queue, but they
    // will be committed in <em>two</em> transactions - one per database.  The SmartClient 
    // server will commit or rollback these two transactions as if they were one, so a 
    // failure in some Oracle update would cause all the updates to both databases to be 
    // rolled back.  However, this is not a true atomic transaction; it is possible for one
    // transaction to be committed whilst the other is not - in the case of hardware failure,
    // for example.
    // <p>
    // NOTE: Not all the supported SQL databases are supported for transactions.  Databases 
    // supported in this release are:
    // <ul>
    // <li>DB2</li>
    // <li>HSQLDB</li>
    // <li>Firebird</li>
    // <li>Informix</li>
    // <li>Microsoft SQL Server</li>
    // <li>MySQL (you must use InnoDB tables; the default MyISAM storage engine does not
    // support transactions)</li>
    // <li>Oracle</li>
    // <li>PostgreSQL</li>
    // </ul>
    //
    // @serverDS only
    // @see OperationBinding.autoJoinTransactions
    // @visibility transactions
    //<
	
	// SQL Paging Strategy
	// ----------------------------------------------------------------------------------------
    //> @type SQLPagingStrategy
    // The technique SmartClient Server's SQL DataSource should use to select a "page" of data 
	// from a table.  
    //
    // @value "sqlLimit"
    //   Specify the paging directly in the SQL query we generate.  The way this is done varies 
	// considerably from database to database: with some it is a straightforward built-in 
	// facility while others require arcane tricks or simply don't support the idea.  This is 
	// the most efficient method, where available.  Note that this strategy is not supported 
	// for operations that make use of a +link{operationBinding.customSQL,customSQL} clause, 
	// because it depends upon being able to determine the size of the whole dataset without
	// actually retrieving the whole dataset.  Ordinary operations do this by means of an 
	// automatically-generated "row count query", but we cannot generate such a query for a 
	// <code>customSQL</code> operation.
    //
    // @value "jdbcScroll"
    //   Implement the paging behavior by use of the <code>absolute()</code> method of the
	// JDBC <code>ResultSet</code>.  
    //
    // @value "dropAtServer"
    //   Implement the paging behavior by fetching the entire resultset from the database and
	// dropping any unnecessary rows on the server before returning the data to the client.
	// This approach is extremely inefficient, but also extremely straightforward; it is 
	// intended as a fallback option, for situations where the more sophisticated approaches
	// cause problems (a JDBC driver that throws vague exceptions when <code>absolute()</code>
	// is called, for example)
    //
    // @value "none"
    //   No paging behavior: we always return the entire resultset to the client.
    //
    // @see dataSource.sqlPaging
    // @see operationBinding.sqlPaging
    // @serverDS only
    // @visibility external
    //<
	
    //> @attr dataSource.sqlPaging (SQLPagingStrategy : null : IRW)
    // The paging strategy to use for this DataSource.  If this property is not set, the 
    // default paging strategy, specified with the +link{server_properties,server.properties}
    // setting <code>sql.defaultPaging</code>, is used.  
	// <p>
	// This setting can be overridden with the +link{OperationBinding.sqlPaging} property.
	// <P>
	// <b>NOTE:</b> Operations that involve a +link{operationBinding.customSQL,customSQL} 
	// clause ignore this property, because customSQL operations usually need to be treated
	// as special cases.  For these operations, the paging strategy comes from the 
	// +link{server_properties,server.properties} setting
	// <code>sql.defaultCustomSQLPaging</code> or 
    // <code>sql.defaultCustomSQLProgressivePaging</code>, depending on whether or not 
    // +link{dataSource.progressiveLoading,progressiveLoading} is in force.  Note that these
	// can always be overridden by a <code>sqlPaging</code> setting on the OperationBinding.
    //
    // @see operationBinding.sqlPaging
    // @serverDS only
    // @visibility external
    //<

    //> @attr dataSource.sqlUsePagingHint (boolean : null : IR)
    // If explicitly set true or left null, causes the server to use a "hint" in the SQL we 
    // generate for paged queries.  If explicitly set false, forces off the use of hints.  This
    // property can be overridden per operationBinding - see 
    // +link{operationBinding.sqlUsePagingHint}.
    // <p>
    // Note this property is only applicable to +link{dataSource.serverType,SQL} DataSources, 
    // only when a +link{dataSource.sqlPaging,paging strategy} of "sqlLimit" is in force,
    // and it only has an effect for those specific database products where we employ a native
    // hint in the generated SQL in an attempt to improve performance.
    //
    // @requiresModules SCServer
    // @see operationBinding.sqlUsePagingHint
    // @group sqlPaging
    // @serverDS only
    // @visibility external
    //<

    //> @attr dataSource.useSubselectForRowCount (boolean : null : IR)
    // This property is only applicable to +link{dataSource.serverType,SQL} DataSources, and 
    // only for +link{class:OperationBinding,operations} that express a 
    // +link{operationBinding.customSQL,customSQL} clause.  In these circumstances, we 
    // generally switch off paging because we are unable to generate a "row count" query that
    // tells the framework the size of the complete, unpaged resultset.
    // <p>
    // The <code>useSubselectForRowCount</code> flag causes the framework to derive a rowcount
    // query by simply wrapping the entire customSQL clause in a subselect, like so:<code><br>
    // &nbsp;&nbsp;&nbsp;&nbsp;SELECT COUNT(*) FROM ({customSQL clause here})</code>
    // <p>
    // However, this is not guaranteed to give good results.  Because the customSQL clause can
    // contain anything that you can write in SQL, running it inside a subselect in order to 
    // count the rows might not work, might have unintended side-effects (if, for example, it 
    // is a stored procedure call that performs updates as part of its function), or it might 
    // just be a bad idea - for example, if the customSQL query is slow-running, you'll make 
    // it twice as slow with this flag, simply because you'll be running it twice.  We 
    // recommend using this flag with care.
    // <p>
    // NOTE: This setting can be overridden per-operation - see 
    // +link{operationBinding.useSubselectForRowCount}.  You can also set a global default for
    // this setting, so you don't have to specify it in every dataSource - define 
    // <code>useSubselectForRowCount</code> as true in your 
    // +link{server_properties,server.properties} file.
    //
    // @requiresModules SCServer
    // @group sqlPaging
    // @serverDS only
    // @visibility external
    //<


    // SQL autoDeriveSchema auto-discovered FK relations
    // ----------------------------------------------------------------------------------------
    
    //> @attr dataSource.tableCode (string : null : R)
    // <b>Only applicable to the built-in SQL DataSource</b>
    // <p>
    // This read-only attribute is a secure and unique hashed version of the name of the table 
    // underlying this dataSource.  It is used automatically by client-side framework code to 
    // link dataSources together by +link{dataSourceField.foreignKey,foreign key}, by matching 
    // DataSource <code>tableCode</code>s with +link{dataSourceField.fkTableCode}s.  This is 
    // necessary if we automatically discovered a foreign key relationship by inspecting the 
    // database schema, as a result of a DataSource specifying +link{autoDeriveSchema,autoDeriveSchema}:
    // true.  We use a secure hash rather than the table name itself for security reasons - 
    // sending the actual table name to the client would be considered an information leakage
    // that could encourage attempts at injection attacks.
    //
    // @requiresModules SCServer
    // @see dataSourceField.fkTableCode
    // @see dataSourceField.fkColumnCode
    // @serverDS only
    // @visibility external
    //<
    
    //> @attr dataSourceField.fkTableCode (string : null : R)
    // <b>Only applicable to the built-in SQL DataSource</b>
    // <p>
    // This read-only attribute is a secure and unique hashed version of the name of the table 
    // underlying a dataSource that this dataSourceField is a 
    // +link{dataSourceField.foreignKey,foreign key} to.  It is used automatically by client-side 
    // framework code to link dataSources together by foreign key, by matching 
    // <code>fkTableCode</code>s with +link{dataSource.tableCode}s.  This is necessary if we 
    // automatically discovered a foreign key relationship by inspecting the database schema, as 
    // a result of a DataSource specifying +link{dataSource.autoDeriveSchema,autoDeriveSchema}:
    // true.  We use a secure hash rather than the table name itself for security reasons - 
    // sending the actual table name to the client would be considered an information leakage 
    // that could encourage attempts at injection attacks.
    //
    // @requiresModules SCServer
    // @see dataSource.tableCode
    // @see dataSourceField.fkColumnCode
    // @serverDS only
    // @visibility external
    //<
    
    //> @attr dataSourceField.fkColumnCode (string : null : R)
    // <b>Only applicable to the built-in SQL DataSource</b>
    // <p>
    // This read-only attribute is a secure and unique hashed version of the name of the matching
    // column in the table underlying a dataSource that this dataSourceField is a 
    // +link{dataSourceField.foreignKey,foreign key} to.  It is used automatically by client-side 
    // framework code to link dataSources together by foreign key.  This is necessary if we 
    // automatically discovered a foreign key relationship by inspecting the database schema, as 
    // a result of a DataSource specifying +link{dataSource.autoDeriveSchema,autoDeriveSchema}:
    // true.  We use a secure hash rather than the column name itself for security reasons - 
    // sending the actual column name to the client would be considered an information leakage 
    // that could encourage attempts at injection attacks.
    //
    // @requiresModules SCServer
    // @see dataSource.tableCode
    // @see dataSourceField.fkTableCode
    // @serverDS only
    // @visibility external
    //<
    
    
    // progressiveLoading
    // ----------------------------------------------------------------------------------------
    
    //> @groupDef progressiveLoading
    // Settings related to the "progressive loading" pattern, which causes data to be loaded 
    // progressively, so that users can request more rows by scrolling off the bottom of the
    // currently visible rows, but cannot cause direct movement to some arbitrary position in 
    // the dataset (as is the case with ordinary, non-progressive loading).  This pattern may
    // give a better user experience with large datasets.
    // @visibility external
    //<
	
    //> @attr dataSource.progressiveLoading (boolean : null : IRW)
    // If true, causes SmartClient Server to use the "progressive loading" pattern for 
    // fetches on this dataSource, as described in the <b>Paging and total dataset length</b>
    // section of the +link{ResultSet,ResultSet documentation}.  Essentially, this means that
    // we avoid issuing a row count query and instead advertise total rows as being slightly 
    // more than the number of rows we have already read (see +link{endGap,endGap}).  This 
    // allows users to load more of
    // a dataset by scrolling past the end of the currently-loaded rows, but it prevents them
    // from scrolling directly to the end of the dataset.
    // <p>
    // Generally, progressive loading is appropriate when you have to deal with very large 
    // datasets. Note that by default, a dataSource will switch into progressive loading mode 
    // automatically when it detects that it is dealing with a dataset beyond a certain size - 
    // see +link{progressiveLoadingThreshold}.
	// <p>
	// This setting can be overridden for individual fetch operations with the 
    // +link{OperationBinding.progressiveLoading} property, and also at the level of the
    // individual +link{DSRequest.progressiveLoading,DSRequest}.  You can also specify 
    // <code>progressiveLoading</code> on 
    // +link{DataBoundComponent.progressiveLoading,DataBoundComponents} and certain types of 
    // <code>FormItem</code> - +link{SelectItem.progressiveLoading,SelectItem} and 
    // +link{ComboBoxItem.progressiveLoading,ComboBoxItem}.
    // <p>
    // Currently, this property only applies to users of the built-in SQLDataSource, but you 
    // could use it in custom DataSource implementations to trigger the server behavior 
    // described in the <code>ResultSet</code> documentation linked to above.
    //
    // @see operationBinding.progressiveLoading
    // @see dataSource.progressiveLoadingThreshold
    // @see dataSource.lookAhead
    // @see dataSource.endGap
    // @serverDS allowed
	// @group progressiveLoading
    // @visibility external
    //<
	
    //> @attr dataSource.progressiveLoadingThreshold (int : 200000 : IRW)
    // Indicates the dataset size that will cause SmartClient Server to automatically
    // switch into +link{progressiveLoading,progressive loading mode} for this DataSource.
    // To prevent automatic switching to progressive loading, set this property to -1.
    //
    // @see dataSource.progressiveLoading
    // @serverDS only
	// @group progressiveLoading
    // @visibility external
    //<
	
    //> @attr dataSource.lookAhead (int : 1 : IRW)
    // If we are +link{progressiveLoading,loading progressively}, indicates the number of 
    // extra records SmartClient Server will read beyond the end record requested by the client,
    // in order to establish if there are more records to view.  This property has no effect 
    // if we are not progressive-loading.
    // <p>
    // This property can be tweaked in conjunction with +link{endGap,endGap} to change behavior
    // at the end of a dataset.  For example, with the default values of <code>lookAhead: 1</code>
    // and <code>endGap: 20</code>, we can end up with the viewport shrinking if we get a case
    // where there really was only one more record (because the client was initially told there
    // were 20 more).  This is not a problem per se, but it may be surprising to the user.
    // You could prevent this happening (at the cost of larger reads) by setting 
    // <code>lookAhead</code> to be <code>endGap+1</code>.
    //
    // @see dataSource.progressiveLoading
    // @see dataSource.endGap
    // @serverDS only
	// @group progressiveLoading
    // @visibility external
    //<
	
    //> @attr dataSource.endGap (int : 20 : IRW)
    // If we are +link{progressiveLoading,loading progressively}, indicates the number of 
    // extra records SmartClient Server will advertise as being available, if it detects that 
    // there are more records to view (see +link{lookAhead,lookAhead}).  This property has no 
    // effect if we are not progressive-loading.
    //
    // @see dataSource.progressiveLoading
    // @see dataSOurce.lookAhead
    // @serverDS only
	// @group progressiveLoading
    // @visibility external
    //<
    

    // cacheAllData implementation
    // ----------------------------------------------------------------------------------------

    //> @attr dataSource.cacheAllData (Boolean : null : IRW)
    // Set this property to true to have a DataSource fetch all of its data client-side on the 
    // first fetch request.  However, unlike a +link{clientOnly} DataSource, this DataSource 
    // will still save changes normally, sending remote requests.
    // <P>
    // You can manually set this attribute after initialization by calling 
    // +link{setCacheAllData}; setting +link{autoCacheAllData}:true causes a DataSource to 
    // automatically switch to <code>cacheAllData:true</code> when a fetch results in the
    // entire dataset being brought client-side.
    // <P>
    // To cause automatic cache updates, you can set +link{cacheMaxAge} to a number of seconds
    // and once data has been client-side for that length of time, the next fetch causes the
    // cache to be dropped and a new cache retrieved.
    // <P>
    // Note that multiple +link{operationBindings} of type "fetch" which return distinct
    // results will not work with <code>cacheAllData</code>: only one cache is created and is
    // used for all fetch operations, regardless of whether +link{dsRequest.operationId} has
    // been set.  However, "fetch" operationBindings used as a
    // +link{operationBinding.cacheSyncOperation} will work normally, so long as they return
    // all data fields that are returned by the default "fetch" operation, so that the cache
    // can be updated.
    // <P>
    // To specify which operationId to use for fetching all data, use
    // +link{cacheAllOperationId}.
    // <p>
    // To use the cache only for requests that have the <code>cacheAllOperationId</code>,
    // allowing any other operationId (or absence of an operationId) to contact the server as
    // normal, set +link{cacheAllOperationIds}.
    //
    // @setter setCacheAllData
    // @group clientData
    // @visibility external
    //<

    //> @attr dataSource.cacheAllOperationId (String : null : IR)
    // +link{dsRequest.operationId} to use for fetching data in case +link{cacheAllData} is
    // true.  By default a standard fetch operation is used (with no <code>operationId</code>
    // specified).
    // @group clientData
    // @visibility external
    //<

    //> @attr dataSource.cacheAcrossOperationIds (Boolean : true : IR)
    // When +link{cacheAllData} mode is enabled and a +link{cacheAllOperationId} has been set,
    // this flag affects whether cached results are used for all "fetch" requests regardless of
    // their +link{dsRequest.operationId}, or are used only for "fetch" requests that use the
    // <code>cacheAllOperationId</code>, allowing other requests to go to server normally.
    // <p>
    // Default of <code>true</code> means that the <code>cacheAllOperationId</code> will be
    // used to fetch all rows, but the resulting cache will be used for all "fetch" operations
    // regardless of the <code>operationId</code> specified on the request.
    // <p>
    // Switching to "false" effectively creates caching just for one specific
    // <code>operationId</code> - the <code>cacheAllOperationId</code> - while allowing all
    // other requests to go to the server normally.
    //
    // @group clientData
    // @visibility external
    //<
    cacheAcrossOperationIds : true,

    //> @method dataSource.setCacheAllData() 
    // Call this method to switch cacheAllData on or off after initialization.  Passing a 
    // <code>shouldCache</code> value of false clears any existing client-side cache,
    // cancels any outstanding requests for a full cache and issues any other pending requests
    // normally.
    // 
    // @param shouldCache (Boolean) New value for +link{cacheAllData}
    // @group clientData
    // @visibility external
    //<
    setCacheAllData : function (shouldCache) {
        if (!shouldCache) {
            if (this.cacheAllData == true) {
                if (this.logIsInfoEnabled("cacheAllData")) {
                    this.logInfo("setCacheAllData(false): clearing the cache and any "+
                        "deferred requests", "cacheAllData");
                }
                // TODO: reset to normal operation
                this.cacheAllData = false;
                delete this._autoCacheAllData_timestamp;
                // 1) clear the cache if there is one
                this.invalidateCache();
                // 2) cancel and ignore any outstanding "fetch" requests for a full cache
                this.clearDeferredRequests();
            }
        } else {
            if (this.logIsInfoEnabled("cacheAllData")) {
                this.logInfo("setCacheAllData(true): invalidate the cache", "cacheAllData");
            }
            this.cacheAllData = true;
            this._autoCacheAllData_timestamp = isc.timestamp();
            this.invalidateCache();
        }
    },

    //> @attr dataSource.cacheMaxAge (Number : 60 : IRW)
    // The maximum time, in seconds, to maintain the client-side cache.  If a fetch occurs after
    // the cacheMaxAge has expired, the current cache will be dropped and another complete
    // cache fetched.
    // @group clientData
    // @visibility external
    //<
    cacheMaxAge: 60,
    // cacheLastFetchTime - the time at which the client-side cache became valid - this value +
    // the cacheMaxAge determines when the cache will be invalidated.
    cacheLastFetchTime: 0,

    cacheNeedsRefresh : function () {
        var currentTime = new Date().getTime(),
            elapsedSeconds = ((currentTime - this.cacheLastFetchTime) / 1000),
            result = (this.cacheLastFetchTime == 0 || elapsedSeconds > this.cacheMaxAge)
        ;

        if (this.logIsInfoEnabled("cacheAllData")) {
            this.logInfo("cacheNeedsRefresh returns "+result, "cacheAllData");
        }

        return result;
    },

    //> @attr dataSource.cacheData (Array of Record : null : IRW)
    // For a +link{cacheAllData} or client-only DataSource, a set of records to use as a dataset, 
    // specified as an Array of JavaScript Objects representing records.
    // @setter setCacheData
    // @group clientData
    // @visibility external
    //<

    //> @method dataSource.setCacheData() 
    // Call this method to set the data in the client-side cache after initialization.
    // @param data (Array of Record) Array of records to apply as the client-side cache
    // @group clientData
    // @visibility external
    //<
    setCacheData : function (data, invalidateCache) {
        if (this.cacheAllData || this.clientOnly) {
            // don't attempt to invalidate the cache if the flag isn't passed
            if (invalidateCache) {
                if (this.logIsInfoEnabled("cacheAllData")) {
                    this.logInfo("setCacheData: invalidating the cache", "cacheAllData");
                }
                // invalidate the cache and clear any pending requests
                this.invalidateCache();
                this.clearDeferredRequests();
            }

            // set the cacheData
            
            this.cacheData = this.testData = data;
            
			// At this point our cacheData is fresh - reset the timer so the 'needsRefresh' method
			// returns false and doesn't ignore it!
			this.cacheLastFetchTime = new Date().getTime();
            
            if (this.logIsInfoEnabled("cacheAllData")) {
                this.logInfo("setCacheData: cacheData has been set", "cacheAllData");
            }
        }
    },

    //> @attr dataSource.testData (Array of Record : null : IRW)
    // For a client-only DataSource, a set of records to use as a dataset, specified as an
    // Array of JavaScript Objects. 
    // Deprecated in favor of +link{dataSource.cacheData}.
    // <P>
    // See +link{group:clientOnlyDataSources,this discussion} for ways to populate a
    // client-only DataSource with test data.
    // @setter setTestData
    // @group clientData
    // @visibility external
    //<

    //> @method dataSource.setTestData()
    // Call this method to set the data in the client-side test-data after initialization. 
    // +link{setCacheData(),setCacheData()} should be called instead and setTestData() is deprecated and will eventually be removed.
    // @param data (Array of Record) Array of records to apply as the client-side test-data
    // @group clientData
    // @visibility external
    //<
    setTestData : function (data, invalidateCache) {
        this.setCacheData(data,invalidateCache);
    },

    //> @attr dataSource.sendParentNode (Boolean : false : IRWA)
    // Set this attribute if you need to send the dsRequest.parentNode to the server-side.
    // @visibility external
    //<
    sendParentNode : false,

    // clear deferred requests - support passing in requestTypes to cancel or "any" to clear
    // all pending requests - if no requestTypes passed, assume "any"
    clearDeferredRequests : function (requestTypes) {
        if (!this._deferredRequests) return;
        requestTypes = requestTypes || "any";
        if (!isc.isAn.Array(requestTypes)) requestTypes = [requestTypes];

        if (this.logIsInfoEnabled("cacheAllData")) {
            this.logInfo("clearDeferredRequests: "+this._deferredRequests.length+" requests, "+
                "clearing those of type "+isc.echoAll(requestTypes), "cacheAllData");
        }

        if (requestTypes.contains("any")) delete this._deferredRequests;
        else {
            if (this._deferredRequests) {
                var requests = this._deferredRequests;
                for (var i=requests.length; i>=0; i--) {
                    var type = requests[i].operationType || "fetch";
                    if (requestTypes.contains(type)) this._deferredRequests.removeAt(i);
                }
                if (this._deferredRequests.length == 0) delete this._deferredRequests;
            }
        }
    },

    // process any deferred requests that are still outstanding
    processDeferredRequests : function () {
        if (!this._deferredRequests) return;
        if (this.logIsInfoEnabled("cacheAllData")) {
            this.logInfo("processDeferredRequests: processing "+this._deferredRequests.length+
                " deferred requests", "cacheAllData");
        }

        var deferredRequests = this._deferredRequests;
        this.clearDeferredRequests();
        // call all deferred requests
        for (var i = 0; i < deferredRequests.length; i++) {
            this.sendDSRequest(deferredRequests[i]);
        }
    },

    //> @method dataSource.invalidateCache() 
    // Invalidate the cache when +link{cacheAllData} or +link{clientOnly} are true.
    // @group clientData
    // @visibility external
    //<
    invalidateCache : function () {
        if (!this.cacheAllData && !this.clientOnly) return;

        if (this.logIsInfoEnabled("cacheAllData")) {
            this.logInfo("invalidateCache: invalidating client-side cache", "cacheAllData");
        }

        delete this.cacheData;
        delete this.testData;
        this.cacheLastFetchTime = 0;
        if (this.cacheResultSet) {
            this.cacheResultSet.destroy();
            delete this.cacheResultSet;
        }
    },

    //> @method dataSource.setClientOnly() 
    // Switch into clientOnly mode, taking the cache from the cacheAllData ResultSet if it 
    // exists.
    // @group clientData
    // @visibility external
    //<
    setClientOnly : function (clientOnly) {
        if (clientOnly) {
            this.clientOnly = true
            if (this.cacheAllData) {
                if (this.cacheResultSet) {
                    if (this.logIsInfoEnabled("cacheAllData")) {
                        this.logInfo("setClientOnly: sourcing from client-cache", "cacheAllData");
                    }
                    this.cacheData = this.testData = this.cacheResultSet.getAllRows();
                }
            } else {
                this.clearDeferredRequests();
                this.invalidateCache();
                this.performDSOperation("fetch");
            }
        }
    },

    //> @method dataSource.hasAllData() 
    // When +link{dataSource.cacheAllData} is true, has all the data been retrieved to the client?
    // @return (Boolean) All data has been fetched from the server and is available client-side
    // @group clientData
    // @visibility external
    //<
    hasAllData : function () {
        if (this.cacheResultSet) return this.cacheResultSet.lengthIsKnown();
        else return false;
    },

    //> @attr dataSource.autoCacheAllData (Boolean : false : IRW)
    // When a DataSource is not +link{dataSource.cacheAllData}:true and a fetch results in the
    // entire dataset being retrieved, this attribute being set to true causes the DataSource
    // to automatically switch to <code>cacheAllData:true</code> and prevent further server-trips for fetch 
    // requests.
    // <p>
    // +link{dataSource.cacheAllData, cacheAllData} is automatically enabled in either of these conditions:
    // <ul>
    //   <li>
    //     The request has no criteria and no startRow/endRow request properties. The latter can
    //     be accomplished by disabling paging with a +link{dataBoundComponent.dataFetchMode,dataFetchMode}
    //     setting of "basic" or "local" or by an explicit fetchData request with those request properties
    //     excluded.
    //  </li>
    //  <li>
    //     The request has no criteria but has startRow/endRow specified and the response received has
    //     all data available (<code>startRow:0</code> and <code>endRow:totalRows</code>).
    //  </li>
    // </ul>
    // @group clientData
    // @visibility external
    //<
    autoCacheAllData: false,

    //> @attr dataSource.useTestDataFetch (boolean : null : IRW)
    // When set, causes a +link{clientOnly, client-only} or +link{cacheAllData} DataSource to 
    // create a second DataSource to perform it's one-time fetch.  By default, this attribute
    // will be considered true when clientOnly is true, cacheAllData is false or unset and
    // a dataURL or testFileName is specified on the DataSource.
    // @group clientData
    // @visibility external
    //<

    
    criteriaContainsDates : function (criteria) {
        var fieldNames = this.getCriteriaFields(criteria);

        if (fieldNames && fieldNames.length > 0) {
            for (var i=0; i<fieldNames.length; i++) {
                var field = this.getField(fieldNames[i]);
                if (!field) continue;
                if (isc.SimpleType.inheritsFrom(field.type, "date") ||
                            isc.SimpleType.inheritsFrom(field.type, "datetime")) {
                    return true;
                }
            }
        }

        return false;
    },

    //> @method dataSource.convertRelativeDates (Criteria : null : IRW)
    // Takes all relative date values found anywhere within a Criteria / AdvancedCriteria object
    // and converts them to concrete date values, returning the new criteria object.
    // @param criteria (Criteria) criteria to convert
    // @param [timezoneOffset] (String) optional timezone offset.  Defaults to the current timezone
    // @param [firstDayOfWeek] (integer) first day of the week (zero is Sunday).  Defaults to
    //                               +link{DateChooser.firstDayOfWeek}
    // @return (Criteria) new copy of the criteria with all relative dates converted
    // @visibility external
    //<
    convertRelativeDates : function (criteria, timezoneOffset, firstDayOfWeek, baseDate) {
        // just bail if passed null criteria
        if (!criteria) return null;

        if (!this.criteriaContainsDates(criteria)) {
            // there are no date fields in the criteria - no point continuing, just return it
            return criteria;
        }

        if (!this.isAdvancedCriteria(criteria) && criteria.operator == null) {
            // this is neither an AdvancedCriteria nor a simple Criterion object so no point
            // parsing it, just return it as-is
            return criteria;
        }

        // get a copy of the criteria to alter and return
        var RD = isc.RelativeDate,
            // ok to use clone() here as we've already confirmed the param is criteria above
            result = isc.clone(criteria);

        baseDate = baseDate || new Date();

        if (firstDayOfWeek == null) firstDayOfWeek = isc.DateChooser ? isc.DateChooser.firstDayOfWeek : 0;

        if (result.criteria && isc.isAn.Array(result.criteria)) {
            // complex sub-criteria, call this method again with that criteria
            var subCriteria = result.criteria;

            for (var i = subCriteria.length-1; i>=0; i--) {
                var subItem = subCriteria[i];

                if (!subItem) {
                    if (this.logIsInfoEnabled("relativeDates")) {
                        this.logInfo("Removing NULL subcriteria...", "relativeDates");
                    }
                    result.criteria.removeAt(i);
                } else {
                    if (subItem.criteria && isc.isAn.Array(subItem.criteria)) {
                        if (this.logIsInfoEnabled("relativeDates")) {
                            this.logInfo("Calling convertRelativeDates from convertRelativeDates "+
                                "- data is:\n\n"+isc.echoFull(subItem)+"\n\n"+
                                "criteria is: \n\n"+isc.echoFull(criteria)
                                ,"relativeDates"
                            );
                        }

                        result.criteria[i] = this.convertRelativeDates(subItem, timezoneOffset,
                            firstDayOfWeek, baseDate);

                        if (this.logIsInfoEnabled("relativeDates")) {
                            this.logInfo("Called convertRelativeDates from convertRelativeDates "+
                            "- data is\n\n" + isc.echoFull(result.criteria[i]), "relativeDates");
                        }
                    } else {
                        result.criteria[i] = this.mapRelativeDate(subItem, baseDate);
                    }
                }
            }
        } else {
            // simple criterion
            result = this.mapRelativeDate(result, baseDate);
        }

        if (this.logIsInfoEnabled("relativeDates")) {
            this.logInfo("Returning from convertRelativeDates - result is:\n\n"+
                isc.echoFull(result)+"\n\n"+
                "original criteria is: \n\n"+isc.echoFull(criteria)
                ,"relativeDates"
            );
        }

        return result;
    },

// helper method to map the relative date in a single criterion
    mapRelativeDate : function (criterion, baseDate) {
        var result = isc.addProperties({}, criterion),
            rangeStart,
            value
        ;

        baseDate = baseDate || new Date();
        
        var fieldName = result.fieldName,
        	field = fieldName ? this.getField(fieldName) : null,
        	type = field ? field.type : null
        ;
        
        
        	
        var logicalDate = isc.SimpleType.inheritsFrom(type, "date") && 
        					!isc.SimpleType.inheritsFrom(type, "datetime");
        

        if (result.value && isc.isAn.Object(result.value) && result.value._constructor == "RelativeDate") 
        {
            // we have a criterion with a "value" and it's a relativeDate - parse it now
            value = result.value.value;
            result.value = isc.DateUtil.getAbsoluteDate(value, baseDate, 
            								result.value.rangePosition, logicalDate || result.value.logicalDate);
        } else {
            if (result.start && isc.isAn.Object(result.start) &&
                result.start._constructor == "RelativeDate")
            {
                // we have a criterion with a "start" and it's a relativeDate - parse it now
                // Always default to start of range for ambiguous shortcuts like "$today"
                value = result.start.value;
                result.start = rangeStart = isc.DateUtil.getAbsoluteDate(value, baseDate,
                									"start", logicalDate || result.start.logicalDate);
            }
            if (result.end && isc.isAn.Object(result.end) && 
                result.end._constructor == "RelativeDate")
            {
                // we have a criterion with an "end" and it's a relativeDate - convert it now
                // Default to end of ambiguous shortcuts like "$today"
                value = result.end.value;
                result.end = isc.DateUtil.getAbsoluteDate(value, baseDate, "end", logicalDate || result.end.logicalDate);
            }
        }

        return result;
    },
    
    //> @attr dataSource.autoConvertRelativeDates (Boolean : true : IR)
    // Whether to convert relative date values to concrete date values before sending to the 
    // server.  Default value is true, which means that the server does not need to understand 
    // how to filter using relative dates - it receives all date values as absolute dates.
    // 
    // @serverDS allowed
    // @visibility external
    //<
    autoConvertRelativeDates: true,

    //> @attr dataSource.showFieldsAsTree (boolean : false : IR)
    // If set, certain components that show lists of fields from this DataSource (including the
    // +link{FieldPicker}) will show them as a tree.
    // <P>
    // The tree structure is created by using +link{tree.modelType,modelType:"parent"} with the
    // +link{dataSourceField.fieldTreeId} and +link{dataSourceField.fieldTreeParentId} properties.
    //<
    showFieldsAsTree: false,

    //> @attr dataSource.fieldTreeRootValue (boolean : null : IR)
    // Value of +link{tree.rootValue}} used if +link{dataSource.showFieldsAsTree} is true
    //<
    
    
    //> @attr dataSource.useAnsiJoins (boolean : null : [IR])
    // For DataSources using the +link{group:sqlDataSource,SmartClient SQL engine} for
    // persistence, whether to use ANSI-style joins (ie, joins implemented with JOIN directives
    // in the table clause, as opposed to additional join expressions in the where clause).
    // The default value of null has the same meaning as setting this flag to false.
    // <P>
    // Note, outer joins (see +link{dataSourceField.joinType,joinType}) only work with certain
    // database products if you choose not to use ANSI joins.  Other than that, the join 
    // strategies are equivalent.
    // <P>
    // If you wish to switch on ANSI-style joins for every DataSource, without the need to 
    // manually set this property on all of them, set
    // +link{server_properties,server.properties} flag 
    // <code>sql.useAnsiJoins</code> to true.
    // 
    // @group serverDataIntegration
    // @requiresModules SCServer
    // @serverDS only
    // @visibility external
    //<
    

    // Used by getFilenameField(), getFilesizeField() and getDateCreatedField()
    _filenameField: {}, 
    _filesizeField: {}, 
    _dateCreatedField: {}


});

//> @type ValueMap
// A ValueMap defines the set of legal values for a field, and optionally allows you to provide
// a mapping from stored values to values as seen by the end user.
// <P>
// A valueMap can be specified as either an Array of legal values, or as an +link{type:Object}
// where each property maps a stored value to a user-displayable value.  See
// +link{DataSourceField.valueMap} for how to express a ValueMap in
// +link{group:componentXML,Component XML}.
// <P>
// A ValueMap can be entirely static or entirely dynamic, with many options in between.  For
// example, a ValueMap may be:
// <ul>
// <li> statically defined in a JavaScript or XML file.  Such a valueMap changes only when
// application code is upgraded.
// <li> generated dynamically by server code when the application first loads, for example,
// by generating JavaScript or XML dynamically in a .jsp or .asp file.  Such a valueMap may be
// different for each session and for each user.
// <li> loaded on demand from a DataSource, via the
// +link{PickList.optionDataSource,optionDataSource} property, or via a call to
// +link{dataSource.fetchData()} where a valueMap is derived dynamically from the returned data
// (see +link{dataSource.fetchData()} for an example).  Such a valueMap may be updated at any
// time, for example, every time the user changes a related field while editing data.
// </ul>
// See also the +link{group:smartArchitecture,SmartClient Architecture Overview} to understand
// the best architecture from a performance and caching perspective.
// @serverDS allowed
// @visibility external
//<

//> @class DataSourceField
// Metadata about a DataSourceField, including its type and validators.
// @treeLocation Client Reference/Data Binding/DataSource
// @serverDS allowed
// @visibility external
//<

//> @groupDef basics
// Basic definition of a DataSource field
// @visibility external
//<

//> @attr dataSourceField.name                 (identifier : null : [IR])
// Name for this field.  
// <p>
// The field name is also the property in each DataSource record which holds the value for this
// field.
// <P>
// Must be unique across all fields within the DataSource as well as a valid JavaScript identifier,
// as specified by ECMA-262 Section 7.6.
// <P>
// <B>NOTE:</B> The <var class="smartclient">+link{String.isValidID()}</var><var class="smartgwt">StringUtil.isValidID()</var>
// function can be used to test whether a name is a valid JavaScript identifier.
//
// @group basics
// @serverDS allowed
// @visibility external
//<

//> @attr dataSourceField.type                 (FieldType : null : [IR])
// Type of this field.  Required for all DataSource fields.
// <P>
// Field type may imply automatic validators (for example, an integer field cannot accept the
// value "foo").  Field type also affects the default behaviors of DataBound components, for
// example, if a field is declared as type "date", components that edit that field will
// automatically choose a date-editing interface with pop-up date picker.
// @group basics
// @serverDS allowed
// @visibility external
//<

//> @attr dataSourceField.mimeType (String : null : IR)
// For a +link{group:binaryFields,binary field}, sets a fixed mime type for all files stored
// to this field.  Should be set to a standard mime type string, for example, "text/plain" for
// a .txt file.
// <P>
// This is useful if a binary field only stored one type of file and you chose not to store 
// filenames, since the extension of the file name is normally used to determine the mime type.
// @serverDS only
// @visibility external
//<

// Validation
// --------------------------------------------------------------------------------------------

//> @attr dataSourceField.required             (boolean : null : [IR])
// Indicates this field must be non-null in order for a record to pass validation.
// <P>
// Note that <code>required</code> should not be set for a server-generated field, such as a
// sequence, or validation will fail on the client.
// @group dataType
// @serverDS allowed
// @visibility external
//<

//> @attr dataSourceField.autoGenerated        (boolean : null : [IR])
// Indicates this field value is auto-generated by data base or ORM provider.
// @group dataType
// @serverDS allowed
// @visibility external
//<

//> @attr dataSourceField.nillable             (boolean : null : IR)
// Controls whether an explicit null-valued Record attribute for this field
// should result in <code>xsi:nil</code> being used to transmit the value when serializing
// to XML, like so:
// <pre>
// &lt;book&gt;
//     &lt;title&gt;Beowulf&lt;/title&gt;
//     &lt;author xsi:nil="true"/&gt;
// &lt;/book&gt;
// </pre>
// If <code>nillable</code> is not set, no XML element will be generated for
// the explicit null value.
// @group dataType
// @serverDS allowed
// @visibility external
//<

//> @attr dataSourceField.valueMap             (ValueMap : null : [IR])
// A +link{type:ValueMap} is a set of legal values for a field.
// <p>
// The valueMap can be specified as either an Array of legal values, or as an
// +link{type:Object} where each property maps a stored value to a user-displayable value.
// <p>
// To enforce that a field should be constrained to only the values in the valueMap, either
// declare +link{dataSourceField.type,field.type} as "enum", or use a
// +link{ValidatorType} of "isOneOf" with explicitly listed values.  Otherwise,
// although a normal +link{SelectItem} control will only allow values from the valueMap to be
// entered, other controls such as a +link{ComboBoxItem,ComboBox} will allow other values 
// to be entered.
// <p>
// In XML, a valueMap that specifies only a list of legal values is specified as follows:
// <pre>
//   &lt;valueMap&gt;
//   	&lt;value&gt;Pens &amp; Pencils&lt;/value&gt;
//   	&lt;value&gt;Stationery&lt;/value&gt;
//   	&lt;value&gt;Computer Products&lt;/value&gt;
//   	&lt;value&gt;Furniture&lt;/value&gt;
//   	&lt;value&gt;Misc&lt;/value&gt;
//   &lt;/valueMap&gt;
// </pre>
// A ValueMap that specifies stored values mapped to user-visible values is specified as
// follows:
// <pre>
//   &lt;valueMap&gt;
//   	&lt;value ID="1"&gt;Pens &amp; Pencils&lt;/value&gt;
//   	&lt;value ID="2"&gt;Stationery&lt;/value&gt;
//   	&lt;value ID="3"&gt;Computer Products&lt;/value&gt;
//   	&lt;value ID="4"&gt;Furniture&lt;/value&gt;
//   	&lt;value ID="5"&gt;Misc&lt;/value&gt;
//   &lt;/valueMap&gt;
// </pre>
// 
// @group dataType
// @serverDS allowed
// @visibility external
//<

//> @attr dataSourceField.valueMapEnum (String : null : IR)
// Fully qualified Java className of an Enum that should be used to derive this field's
// +link{valueMap}.
// <P>
// This has the same behavior of auto-deriving a <code>valueMap</code> from a Java Enum as
// +link{DataSource.autoDeriveSchema} except it applies only to one field.
// <P>
// If you enable <code>autoDeriveSchema</code> there is no need to set
// <code>valueMapEnum</code> for Enum fields unless you want to actually <i>override</i> the
// behavior to use a different Enum for the field (eg, a superclass Enum that is more
// restrictive).
//
// @serverDS only
// @visibility external
//<

//> @attr dataSourceField.validators           (Array of Validator : null : [IR])
// Validators to be applied to this field.
// <p>
// Validators are applied whenever there is an attempt to save changes to a field.
// <p>
// For the available set of built-in validators, and how to define a custom validator, see the
// +link{class:Validator} class.
//
// @see class:Validator
// @group dataType
// @group validation
// @serverDS allowed
// @visibility external
//<

//> @attr dataSourceField.length               (number : null : [IR])
// Maximum number of characters allowed.  Applicable only to fields of text type.
// <P>
// <b>NOTE:</b> For DataSources of type "sql", this property has a bearing on the type of 
// column we use when the underlying table is created by a DataSource 
// +link{group:sqlDataSource,import} in the +link{group:adminConsole,Admin Console}.  Below 
// a certain length (which is database-specific, see below), we use standard <code>VARCHAR</code>
// columns; above that length, we use an alternate strategy (again, database-specific).  For 
// these long fields, we sometimes also generate different SQL for "update" and "add" 
// operations, using JDBC "?" replacement parameters rather than embedding values directly in 
// the generated SQL; whether or not this is done depends entirely on what the underlying 
// database product and/or JDBC driver will allow.
// <P><br>
// <b>Table of field length limits for supported databases:</b><p>
// <table style="font-size:10px;text-align:center;border:1px solid black;">
// <tr><td style="color:white;background-color:black;"><b>Database product</b></td>
//     <td style="color:white;background-color:black;"><b>VARCHAR limit *</b></td>
//     <td style="color:white;background-color:black;"><b>Type used above limit</b></td></tr>
// <tr><td>HSQLDB</td><td>None</td><td>-</td></tr>
// <tr><td>IBM DB2</td><td>4000</td><td>CLOB</td></tr>
// <tr><td>Firebird</td><td>32767</td><td>BLOB with subtype 1</td></tr>
// <tr><td>Informix</td><td>255 / 32739</td><td>LVARCHAR / TEXT ** </td></tr>
// <tr><td> Microsoft SQL Server </td><td>8000</td><td>TEXT</td></tr>
// <tr><td>MySQL</td><td> 255 / 65535 / 16M </td><td> TEXT / MEDIUMTEXT / LONGTEXT *** </td></tr>
// <tr><td>Oracle</td><td>4000</td><td>CLOB</td></tr>
// <tr><td>PostgreSQL</td><td>4000</td><td>TEXT</td></tr>
// </table><br>
// <b>*</b> The "VARCHAR limit" referred to here is a limit used by the SmartClient Server; it
// is not necessarily imposed by the database.  For example, DB2's VARCHAR limit is not 4000
// characters; it actually varies from about 4K to about 32K, depending on how the server has 
// been configured.<p>
// <b>**</b> Informix has a limit of just 255 characters for VARCHAR, but has a native LVARCHAR
// type which supports nearly 32K characters without needing to fall back on long datatypes. 
// Therefore, with that one product, we have two thresholds for a change in storage type.<p>
// <b>***</b> MySQL has a limit of 255 characters for VARCHAR, 65,535 characters for TEXT and 
// 16,777,215 for MEDIUMTEXT; therefore, with that one product, we have three thresholds for a 
// change in storage type.
// @group dataType
// @serverDS allowed
// @visibility external
// @example longText
//<

// XML
// ---------------------------------------------------------------------------------------

//> @attr dataSourceField.valueXPath      (XPathExpression : null : [IR])
// XPath expression used to retrieve the field's value.
// <P>
// This XPath expression will be evaluated in the scope of the record objects selected by 
// the +link{dataSource.recordXPath}.  For XML data 
// (+link{DataSource.dataFormat,dataFormat:"xml"}) this means a call to
// +link{XMLTools.selectString()} passing the selected XML element.  For JSON data
// (+link{DataSource.dataFormat,dataFormat:"json"}), this means a call to 
// +link{XMLTools.selectObjects()} passing the selected JSON object.
// <P>
// In the absence of a <code>valueXPath</code>, for JSON data the value for the field will
// be the value of the same-named property in the record object selected by
// +link{DataSource.recordXPath,recordXPath}.  
// <P>
// For XML data, the value will be the attribute or subelement named after the field name.
// For example, for a field "author" on a record element &lt;book&gt;, the following structures
// require no valueXPath:
// <pre>
//    &lt;book author="Mark Jones"/&gt;
//
//    &lt;book&gt;
//        &lt;author&gt;Mark Jones&lt;/author&gt;
//    &lt;/book&gt;
// </pre>
// <P>
// If <code>valueXPath</code> is not required for your field because of the default handling
// described above, don't specify it, as it's slightly slower.
// <p>
// To learn about XPath, try the following search:
// <a href="http://www.google.com/search?q=xpath+tutorial" target="_blank"
// >http://www.google.com/search?q=xpath+tutorial</a>
// <P>
// <B>Using valueXPath with the SmartClient server</B>
// <p>
// If you're using the SmartClient server to return data via the DSResponse object (or
// indirectly doing so using DataSource DMI), the valueXPath you specify on the DataSource
// fields will be applied to the data you return via the 
// +externalLink{http://commons.apache.org/jxpath/,JXPath} library.
// <P>
// If you are returning Java Beans as your DSResponse data, normally each dataSource field
// receives the value of the same-named Java Bean property, that is, a field "zipCode" is
// populated by looking for "getZipCode()" on the objects passed as DSResponse data.  You can
// use <code>valueXPath</code> to retrieve properties from subobjects, so long as a chain of
// getter methods exists that corresponds to the valueXPath.  For example, a valueXPath of
// "address/zipCode" expects to call "getAddress()" on the bean(s) passed to
// DSResponse.setData(), followed by "getZipCode()" on whatever object "getAddress()" returns.
// <P>
// When you are saving data, the inbound DSRequest values, available as a Java Map, will use 
// just dataSource field names as Map keys, not the valueXPath used to derive them.  However,
// to achieve bidirectional valueXPath binding, you can use the server-side method
// dataSource.setProperties() to use the valueXPath when setting properties on your server
// object model.  When applied as a setter, an XPath like "address/zipCode" attempts
// "getAddress()" followed by "setZipCode()" on the returned object.  JXPath also has some
// ability to auto-create intervening objects if they are missing, such as auto-creating an
// "address" subobject when applying "address/zipCode" as a valueXPath.
// <P>
// See the
// +externalLink{http://jakarta.apache.org/commons/jxpath/,JXPath} library documentation for
// complete details, including other types of server object models supported, such as
// server-side XML.
//
// @see DataSource.setProperties() (Java API)
// @see DSResponse.setData() (Java API)
// @group clientDataIntegration
// @serverDS allowed
// @visibility xmlBinding
// @example xpathBinding
//<

//> @attr dataSourceField.valueWriteXPath      (XPathExpression : null : [IR])
// Alternative XPath expression used to set the field's value.
// <P>
// If is not set, then +link{DataSourceField.valueXPath,dataSourceField.valueXPath} is
// used, see its description for details.
//
// @see DataSource.setProperties() (Java API)
// @see DSResponse.setData() (Java API)
// @group clientDataIntegration
// @serverDS only
// @visibility xmlBinding
//<

//> @method dataSourceField.getFieldValue() [A]
// Function or JavaScript expression used to retrieve the field's value from the XML element or
// JSON record returned from a web service.
// <P>
// This is an advanced API for use when a +link{dataSourceField.valueXPath,valueXPath} setting
// is insufficient to derive a field's value, yet an implementation of
// +link{DataSource.transformResponse()} is overkill.
//
// @param record (Object or XMLElement) record object selected from web service response data
//                                      by +link{operationBinding.recordXPath,recordXPath}
// @param value (any) default value derived by the method described in
//                    +link{dataSourceField.valueXPath}
// @param field (DataSourceField) DataSourceField definition
// @param fieldName (String) name of the DataSource field
//
// @group clientDataIntegration
// @serverDS allowed
// @visibility xmlBinding
//<

//> @attr dataSourceField.lenientXPath (boolean : null : IR)
// Indicates that getting +link{dataSourceField.valueXPath} for this field should
// not perform any validation at all and will return null for non existing XPaths.
// Otherwise warning message will be logged for non-existing XPath or with null objects
// in the middle of XPath.
// <P>
// NOTE: this applies to server-side processing of valueXPath only.
//
// @serverDS allowed
// @visibility external
//<

// Component Binding
// --------------------------------------------------------------------------------------------

//> @groupDef componentBinding
// Properties that control how a DataBound Component binds to this DataSource
// @title Component Binding
// @visibility external
//< 

//> @attr dataSourceField.title                (String : null : [IR])
// Default user-visible title for this field.
// <p>
// This will be picked up by DataBound components and other views over this DataSource.
// <p>
// Note this property frequently does not need to be set since +link{DataSource.autoDeriveTitles} (on by
// default) usually picks an appropriate user-visible title if you have a reasonable naming convention
// for your field names.
// <P>
// Note that if this field is being displayed in a +link{ListGrid} bound to this dataSource,
// the +link{listGridField.headerTitle} attribute may be used to specify a
// different string for display in the listGrid column header.
//
// @group componentBinding
// @serverDS allowed
// @visibility external
//<

//> @attr dataSourceField.escapeHTML	(Boolean : null : [IR])
// When data values are displayed in DataBound components, by default strings will be interpreted
// as HTML by the browser in most cases.
// <P>
// If set, this property will be picked up by components bound to this dataSource, notifying them
// that any HTML characters should be escaped when displaying values for this field. 
// @see ListGridField.escapeHTML
// @serverDS allowed
// @visibility external
//<

//> @attr dataSourceField.canView              (boolean : null : [IR])
// If false, this property indicates that this field is considered "server only".  This means:
// <ul>
// <li>Components cannot bind to the field; even if you explicitly add a field with the same 
//     name to your +link{DataBoundComponent,dataBoundComponent}, it will be dropped</li>
// <li>If you are using SmartClient Server, the client will never be sent a value for the 
//     field</li>
// <li>If you are using SmartClient Server, then similar to +link{canEdit,canEdit}, no updates
//     to the field are allowed from the client.  If you explicitly add a value for the field 
//     to, eg, a record you are passing to +link{dataSource.updateData}, the server will strip
//     the value out of the record before processing the update request.
// </ul>
// <code>canView:false</code> is <b>not</b> the same thing as +link{hidden,hidden}.  Use 
// <code>canView:false</code> when you want to prevent the client from ever seeing a field's 
// definition or values; use <code>hidden:true</code> if it is fine from a security perspective
// that a field's definition and values are sent to the browser, but the field should not by
// default appear in user interface elements (but could do in some cases, like a special screen
// for advanced users or administrators, for example).
// <p>
// Note that this property must be set explicitly to false to have an effect; a null or 
// undefined setting is treated the same as true.
// <P>
// This property is used to implement field-level view security: failing a 
// +link{viewRequiresAuthentication,viewRequiresAuthentication}, 
// +link{viewRequiresRole,viewRequiresRole} or +link{viewRequires,viewRequires} test is 
// equivalent to setting <code>canView:false</code> on the field (and, indeed, from the 
// client's perspective, the field <em>has</em> had <code>canView:false</code> set).
// @group componentBinding
// @serverDS allowed
// @visibility external
//<

//> @attr dataSourceField.canEdit              (boolean : null : [IR])
// Controls whether, by default, DataBoundComponents consider this field editable. 
// Set to <code>false</code> to draw this field read-only.  
// <p>
// For a field that should never be changed from the UI, consider using +link{canSave} instead,
// which will both mark a field as non-editable and reject any attempt to programmatically
// change the value on the server (when using the SmartClient Server).
// <P>
// This attribute may not effect all dataBoundComponents - the 
// +link{databoundComponent.canEditFieldAttribute} may be set at the component level to look
// for a different attribute on the dataSourceField, and components allow developers to explicitly
// override this default (see +link{ListGridField.canEdit}. +link{FormItem.canEdit} for example).
//
// @group componentBinding
// @see dataSourceField.canFilter
// @see dataSourceField.canSave
// @serverDS allowed
// @visibility external
//<

//> @attr dataSourceField.canSave (boolean : null : [IR])
// Whether values in this field can be updated and saved to the dataSource.
// <P>
// If set to <code>false</code>, this field will default to being non-editable in standard editing
// components (+link{DynamicForm}, editable +link{ListGrid}), but will be editable when displayed
// for filtering purposes only (in a +link{SearchForm} or +link{ListGrid.showFilterEditor,ListGrid
// filter editor}.  If +link{dataSourceField.canEdit,canEdit} is explicitly specified it will take 
// precedence over this client-side behavior, but the server will still enforce the no-save
// policy (described below).
// <p>
// NOTE: If you are using SmartClient Server and have specified <code>canSave: false</code> 
// for a field in a DataSource definition (<code>.ds.xml</code> file), this is enforced on 
// the server.  This means that we will strip out any attempt to set the value of such a 
// field before trying to process any update or add request, similar to what happens when 
// a +link{editRequiresAuthentication,field-level declarative security check} fails.
// @group componentBinding
// @serverDS allowed
// @visibility external
//<

//> @attr dataSourceField.canFilter (boolean : null : IR)
// Should the user be able to filter data by this field?
// Effects whether this field will show up in dataBoundComponents with UI for filtering data.
// <P>
// @see SearchForm.showFilterFieldsOnly
// @see SearchForm.canEditFieldAttribute
//
// @serverDS allowed
// @visibility external
//<

//> @attr dataSourceField.editorType           (FormItem className : null : [IR])
// Sets the default FormItem to be used whenever this field is edited (whether in a grid, form,
// or other component).
// <P>
// If unset, a FormItem will be automatically chosen based on the type of the field, by the
// rules explained +link{type:FormItemType,here}.
// 
// @group componentBinding
// @serverDS allowed
// @visibility external
//<

//> @attr dataSourceField.readOnlyEditorType           (FormItem className : null : [IR])
// Sets the default FormItem to be used if this field is marked as 
// +link{dataSourceField.canEdit,canEdit false} and displayed in an editor component such
// as a DynamicForm.
// <P>
// This property may also be specified at the type level by specifying 
// +link{SimpleType.readOnlyEditorType}.
// 
// @group componentBinding
// @serverDS allowed
// @visibility external
//<

//> @attr dataSourceField.displayFormat        (DateDisplayFormat : null : [IR])
// The default date formatter to use for displaying this field.  Only applicable to fields of 
// type "date" and "datetime".  Note that this property is honored when exporting directly to 
// Excel spreadsheets (ie, when using XLS or XLSX/OOXML form, <b>not</b> CSV); "date" and
// "datetime" fields with this property set will deliver real dates and formatting information
// to Excel, rather than formatted strings or unformatted dates.
// 
// @serverDS allowed
// @visibility external
// @deprecated Use +link{dataSourceField.dateFormatter} or +link{dataSourceField.timeFormatter}
//   instead.
//<

//>	@attr dataSourceField.dateFormatter (DateDisplayFormat : null : [IRWA])
// Preferred display format to use for date type values within this field.
// If this property is set on a field displayed in a databound component such as a
// +link{DynamicForm} or +link{ListGrid} it will be respected (See +link{formItem.dateFormatter} and
// +link{listGridField.dateFormatter}).
// <P>
// Note that this property is also honored when exporting directly to 
// Excel spreadsheets (ie, when using XLS or XLSX/OOXML form, <b>not</b> CSV); "date" and
// "datetime" fields with this property set will deliver real dates and formatting information
// to Excel, rather than formatted strings or unformatted dates.
// 
// @group appearance
// @visibility external
//<
//dateFormatter:null

//>	@attr dataSourceField.timeFormatter (TimeDisplayFormat : null : [IRWA])
// Preferred time-format to apply to date type values within this field.  If this property is
// specified on a field displayed within a dataBound component such as a +link{listGrid} or
// +link{dynamicForm}, any dates displayed in this field will be formatted as times using the
// appropriate format.
// <P>
// This is most commonly only applied to fields specified as type <code>"time"</code> though
// if no explicit +link{formItem.dateFormatter} is specified it will be respected for other 
// fields as well.
// <P>
// See +link{listGridField.timeFormatter} and +link{formItem.timeFormatter} for more information.
//
// @group appearance
// @visibility external
//<
//timeFormatter:null

//> @attr dataSourceField.decimalPrecision (number : null : [IRW])
// Applies only to fields of type "float" and affects how many significant digits are shown.
// <P>
// For example, with decimalPrecision 3, if the field value is 343.672677, 343.673 is shown.
// <P>
// If the value is 125.2, 125.2 is shown - decimalPrecision will not cause extra zeros to be
// added.  Use +link{dataSourceField.decimalPad} for this.
// <P>
// A number is always shown with its original precision when edited.
//
// @group appearance
// @serverDS allowed
// @visibility external
//<

//> @attr dataSourceField.decimalPad (number : null : [IRW])
// Applies only to fields of type "float" and enforces a minimum number of digits shown after
// the decimal point.
// <P>
// For example, a field value of 343.1, 343.104 and 343.09872677 would all be shown as 343.10
// if decimalPad is 2.
// <P>
// The original unpadded value is always shown when the value is edited.
//
// @group appearance
// @serverDS allowed
// @visibility external
//<

//> @attr dataSourceField.precision (number : null : [IRW])
// Applies only to fields of type "float" or "integer" and affects how many significant
// digits are shown.
// <P>
// For example, a field value of 343.1, 343.104 and 343.09872677 would all be shown as 343.10
// if precision is 5.
// <P>
// The original value is always shown when the value is edited.
//
// @group appearance
// @serverDS allowed
// @visibility external
//<

//> @attr dataSourceField.hidden (boolean : false : [IR])
// Whether this field should be hidden from users by default within a DataBound component.
// This is generally used for internal IDs and other fields not meaningful to users.
// <P>
// See +link{dataSourceField.detail} for fields that should be hidden in a summary view such as
// a +link{ListGrid}, but still available to the user.
// <p>
// <b>NOTE:</b> This property is <b>not</b> a security setting - data for hidden fields is 
// still delivered to the client, it just isn't shown to the user.  If you wish to make sure 
// that only appropriate data reaches the client, use +link{operationBinding.outputs},
// +link{canView}:false on the field, or a field-level declarative security setting like 
// +link{dataSourceField.viewRequiresRole}.
// 
// @group componentBinding
// @serverDS allowed
// @visibility external
//<

//> @attr dataSourceField.ignore (boolean : false : [IR])
// Whether this field should be completely excluded from this dataSource, as if it had never been
// defined.
// <P>
// If set to true, the field will be entirely omitted when serving a DataSource derived 
// from a server-side definition (typically a .ds.xml file) to the client.
// 
// @group componentBinding
// @serverDS only
// @visibility external
//<

//> @attr dataSourceField.detail (boolean : false : [IR])
// Whether this field should be considered a "detail" field by a +link{DataBoundComponent}.
// <P>
// Detail fields won't be shown by default in a DataBoundComponent where 
// +link{DataBoundComponent.showDetailFields} is false.  This allows for some DataBound
// components, like a +link{ListGrid}, to show a summary view of records which displays only 
// the most commonly viewed fields by default, while other DataBoundComponents, like a
// +link{DetailViewer}, show all fields by default.
// <P>
// In addition, the +link{formItem.showIf,formItem.showIf} property is supported in multiple
// components for conditional visibility - see for example +link{listGridField.showIf} and
// +link{formItem.showIf}).
//
// @group componentBinding
// @serverDS allowed
// @visibility external
//<

//> @attr dataSourceField.excludeFromState (Boolean : null : IR)
// If true, then this field is excluded from the bound component's view state. In addition,
// the field will not be selected as the default title field
// by +link{DataBoundComponent.getTitleField()}
// if +link{DataBoundComponent.titleField} is not provided.
// @group viewState
//<


// Relations
// --------------------------------------------------------------------------------------------

//> @groupDef dataSourceRelations
// Properties for declaring relations between DataSources and within DataSources, such as
// master-detail relationships (a SalesOrder contains 0 or more OrderItems) and tree
// relationships (records can be linked by a parentId field).
//
// @see DataSourceField.foreignKey
// @title Relations
// @visibility external
//<

//> @attr dataSourceField.primaryKey           (boolean : false : [IR])
// Indicates <b>either</b> that this field holds a value unique across all records in this
// DataSource, <b>or</b> that it is one of a number of fields marked as primary keys, and the
// combination of the values held in all of those fields is unique across all records in the
// DataSource.  Note that the latter usage - so-called "composite" or "multipart" keys - is 
// intended for support of legacy databases only: if you are able to choose an approach, 
// Isomorphic recommends the use of one <code>primaryKey</code> field per DataSource, and 
// ideally this field should be of +link{dataSourceField.type,type} "sequence".
// <p>
// A DataSource that can <i>only</i> perform the "fetch" operation does not require a
// primaryKey unless it contains +link{group:binaryFields,binary fields}.  If a DataSource 
// allows modification of DataSource records through add, update and remove DataSource 
// operations, or it contains one or more binary fields, one or more fields must be marked as 
// the primary key.
// <P>
// SmartClient requires a primary key value to uniquely identify records when communicating
// updates or deletions to the server.  There is no requirement that the primaryKey field be
// mapped to an actual "primary key" in your object model, web service, or database (though 
// this is the most obvious and natural thing to do, of course).  The only requirement is 
// that the combined values of the primaryKey fields be unique for a given browser instance
// for the lifetime of the page.  
// <P>
// If using SmartClient's +link{group:sqlDataSource,SQL engine} and generating SQL tables using
// the +link{group:adminConsole,Admin Console}, the table column generated from a primaryKey 
// field will have a unique constraint applied in the database table and, if the field is of 
// type "sequence", the database column will also be created as an "identity column" in those 
// databases that implement sequence-type handling with identity columns.
//
// @group dataSourceRelations
// @serverDS allowed
// @visibility external
//<

//> @attr dataSourceField.foreignKey           (String : false : [IR])
// Declares that this field holds values that can be matched to values from another DataSource
// field, to create a relationship between records from different DataSources or even records
// within the same DataSource.
// <p>
// The format of <code>foreignKey</code> is
// <code>&lt;dataSourceId&gt;.&lt;fieldName&gt;</code>.<br>
// For a foreignKey within the same dataSource, you can omit the <code>dataSourceId</code>
// and just specify <code>&lt;fieldName&gt;</code>.<br>
// For example, to create a tree relationship within a DataSource:
// <pre>
//   isc.DataSource.create({
//     ID:"supplyItem",
//     fields : [
//       {name:"itemId", type:"sequence", primaryKey:true},
//       {name:"parentId", type:"integer", foreignKey:"itemId"},
//       ...
//     ]
//   });
// </pre>
// <P>
// <code>foreignKey</code> declarations also allow other automatic behaviors by
// +link{DataBoundComponent,DataBoundComponents}, such as +link{listGrid.fetchRelatedData()}.
// <p>
// For SQLDataSources foreign keys can be automatically discovered from SQL tables if
// +link{dataSource.autoDeriveSchema,autoDeriveSchema} is set.
//
// @see dataSourceField.joinType
// @group dataSourceRelations
// @serverDS allowed
// @visibility external
//<

//> @attr dataSourceField.childrenProperty (Boolean : false : [IR])
// If true, this property indicates that this field will hold an explicit array of child nodes 
// for the current node.
// This has the same effect as specifying +link{dataSource.childrenField} to this field's 
// name.
// @see dataSource.childrenField
// @group dataSourceRelations
// @serverDS allowed
// @visibility external
// @example treeLoadXML
//<


//> @attr dataSourceField.rootValue            (any : null : [IR])
// For a field that is a foreignKey establishing a tree relationship, what value indicates a
// root-level node.  Defaults to null.
// <P>
// Note that the rootValue may be overridden on a specific ResultTree instance by setting
// +link{ResultTree.rootNode}, or if the ResultTree is auto-generated by a +link{TreeGrid},
// by setting +link{TreeGrid.treeRootValue}. This allows a component to navigate a subtree of
// the hierarchical data from this dataSource starting at a particular node.
// @group dataSourceRelations
// @serverDS allowed
// @visibility external
// @example treeLoadXML
//<

//> @attr dataSourceField.sequenceName (String : null : IR)
// For a DataSource with +link{dataSource.serverType,serverType:"sql"} with a field of type
// "sequence", the name of the SQL sequence that should be used when inserting new records into
// this table.
// <P>
// Note that this is never required for SQL tables that are generated from SmartClient
// DataSources (a default sequence name of tableName + "_" + columnName is chosen, but see the
// notes below regarding this), and is never required for databases where inserting null into a 
// sequence column is sufficient (MySQL, SQL Server, DB2 and others). 
// <P>
// You would only need to set sequenceName if you are integrating with a pre-existing table
// stored in a database where the sequence must be named for insertion to work (Oracle,
// Postgres, Firebird) OR you are trying to use the same sequence across multiple DataSources.
// <P>
// <b>Note:</b> If you specify the <code>sql.{database type}.sequence.name.prefix</code> and/or 
// <code>sql.{database type}.sequence.name.suffix</code> properties in your 
// +link{server_properties,server.properties} file,the generated sequence name will include the prefix and/or
// suffix.  For example, with a prefix of "order_system_" and a suffix of "_seq", the sequence
// generated for column "orderNumber" on table "orders" would be:<pre>
//     order_system_orders_orderNumber_seq
// </pre>
//
// @group sqlDataSource
// @serverDS allowed
// @visibility external
//<

//> @attr dataSourceField.includeFrom (String : null : IR)
// Indicates this field should be fetched from another, related DataSource.
// <P>
// The <code>incluedFrom</code> attribute should be of the form
// "<i>dataSourceId</i>.<i>fieldName</i>", for example:
// <pre>
//    &lt;field includeFrom="supplyItem.itemName"/&gt;
// </pre>
// <P>
// A +link{foreignKey} declaration must exist between the two DataSources, establishing either
// a 1-to-1 relationship or a many-to-1 relationship from this DataSource to the related
// DataSource.  The inclusion can be indirect (traverse multiple DataSources) so long as there
// is a chain of <code>foreignKey</code> declarations from the target DataSource to the
// DataSource where the <code>includeFrom</code> field is declared.
// <P>
// +link{DataSourceField.name} will default to the name of the included field, or you can
// specify a different name.
// <p>
// If both DataSources are SQLDataSources, HibernateDataSources or JPADataSources (with
// Hibernate as the provider) the related data will be retrieved via a SQL join and criteria
// and sort directions applied to the field work normally (they become part of the generated
// SQL query).
// <P>
// Otherwise, the related data will be retrieved via performing a DSRequest against
// the related DataSource once the data from the primary DataSource has been retrieved.  In
// this case, criteria or sorting directions applied to the included field are only allowed if
// data paging is not in use (for example +link{listGrid.dataFetchMode}:"basic"); otherwise,
// criteria and sort direction are ignored for the included field and a warning is logged on
// the server.
// <P>
// <b>Editing included fields</b>
// <p>
// An included field is +link{dataSourceField.canEdit,canEdit:false} by default.  Note that 
// included fields are not updatable, even if you set canEdit:true; the server will simply drop
// values for included fields if client code sends them. 
// <p>
// When thinking about editing an included field value, typically what is really intended is to
// edit the value of the <code>foreignKey</code> field.  For example, take the scenario of a
// system that tracks accounts and the employees assigned to manage them.  Given a DataSource
// "account" related one-to-one with DataSource "employee" by a "managerId" foreignKey field,
// we might declare an <code>includeFrom</code> so that the name of the account manager can
// be shown with each "account" record.  
// <p>
// Editing the manager's name while viewing the account would be intended to pick a new account
// manager, and <b>not</b> to change the legal name of the employee who happens to be the
// current account manager. 
// <p>
// To correctly set up this scenario, declare an <code>includeFrom</code> field that is hidden,
// but is used as the +link{dataSourceField.displayField,displayField} for the foreign key
// field:
// <pre>
// &lt;field name="managerId" foreignKey="employee.id" displayField="managerName"/&gt; 
// &lt;field name="managerName" includeFrom="employee.name" hidden="true"/&gt; 
// </pre>
// Now:
// <ul>
// <li> the "managerId" foreignKey field is shown in grids and forms, but takes its displayed
// value from the hidden <code>includeFrom</code> field
// <li> the automatically chosen editor will be a SelectItem with
// +link{SelectItem.optionDataSource,optionDataSource} set to "employees": it will allow
// picking a different "employee" record from the "employee" DataSource
// <li> saving will save the ID of a new "employee" record to the "managerId" foreign key
// field, as intended
// </ul>
// You can alternatively set <code>editorType="ComboBoxItem"</code> on the
// "managerId" field to allow typeahead search of the "employee" DataSource.
//
// @example sqlIncludeFrom 
// @serverDS only
// @group dataSourceRelations
// @visibility crossDS
//<


//> @attr dataSourceField.displayField (String : null : IR)
// Name of another field in this DataSource that should be used as the display value for this
// field.
// <P>
// Typically used for editable 
// +link{DataSourceField.foreignKey,foreignKey} fields: the <code>foreignKey</code> field
// stores an ID value, and this ID value is the right value to use when editing (typically by a
// +link{SelectItem} with +link{SelectItem.optionDataSource,optionDataSource} set).  However,
// when the <code>foreignKey</code> field is viewed read-only, it should display a name, title
// or other friendly value from the related record.  In order to accomplish this, a second,
// hidden field carries the display value, and the <code>foreignKey</code> field has
// <code>displayField</code> set to this second, hidden field.
// <P>
// For a more in-depth discussion, see +link{dataSourceField.includeFrom}.
//
// @serverDS allowed
// @group dataSourceRelations
// @visibility external
//<

//> @attr dataSourceField.joinType (JoinType : null : IR)
// This property is only applicable to fields of SQL DataSources that also specify a 
// +link{foreignKey,foreignKey} property; it is ignored for all other fields.  Indicates the 
// type of join to make between the tables underlying this DataSource and the other DataSource
// referred to in the <code>foreignKey</code> property, when resolving +link{includeFrom,includeFrom} 
// fields.  The default value of null is the same as specifying "inner".
// <P>
// Note, outer joins are allowed for all supported database products only if you are using 
// +link{dataSource.useAnsiJoins,ANSI-style joins}, which is not the case by default.  If you
// are using the older strategy of additional join expressions in the WHERE clause, outer 
// joins are only supported for database products that provide a proprietary native syntax for 
// expressing outer joins.  Those products are:<ul>
// <li>Oracle</li>
// <li>Versions of Microsoft SQL Server earlier than 2012, and running in compatibility mode 80</li>
// </ul>
//
// @serverDS only
// @group dataSourceRelations
// @visibility external
//<


//> @type JoinType
// The type of join to make between two SQL tables when resolving 
// +link{dataSourceField.includeFrom,includeFrom} fields.  
//
// @value "inner"    A regular inner join, whereby rows are only included in the resultset
//                   where the join can be satisified, so a missing row in the table being
//                   joined to results in the entire row being omitted.
// @value "outer"    An outer join.  All outer joins generated by SmartClient's SQL subsystem
//                   are left outer joins, meaning that every row in the join-from (or "left")
//                   table that matches the criteria is included, and missing rows in the 
//                   join-to (or "right") table cause columns to be set to null.
//
// @serverDS only
// @group dataSourceRelations
// @visibility external
//<


// Summary functions

//> @attr dataSourceField.summaryFunction (SummaryFunction : null : IR)
// @include listGridField.summaryFunction
// @visibility external
//<

//> @attr dataSourceField.summaryValueTitle (String : null : IR)
// Title to show in a +link{SummaryFunction,Summary of type "title"} for this field. If unspecified
// <code>title</code> summaries will show the +link{dataSourceField.title} for the field.
// @visibility external
//<

// XML binding / serialization
// ---------------------------------------------------------------------------------------

//> @attr dataSourceField.xmlAttribute (boolean : null : IR)
// Indicates that +link{dataSource.xmlSerialize()} should serialize this value as an XML
// attribute.
// <P>
// Note this does not need to be declared in order for DataSource records to be derived from
// XML data: a field will be populated with either an attribute or subelement with matching
// name.
//
// @group xmlSerialize
// @group componentSchema
// @serverDS allowed
// @visibility external
//<

//> @attr dataSourceField.multiple (boolean : null : IR)
// Indicates that this field should always be Array-valued.  If the value derived from
// +link{dataSource.dataFormat,XML or JSON data} is singular, it will be wrapped in an Array.
// <P>
// <h4>Criteria on multiple:true fields: client-side filtering</h4>
// <p>
// For simple Criteria, the criteria value is compared to <i>each</i> field value in the
// <code>multiple:true</code> field, according to the
// +link{dsRequest.textMatchStyle,textMatchStyle}.  If <i>any</i> field value matches the
// filter value, the field is considered to match the criteria.
// <p>
// For +link{AdvancedCriteria}, for normal +link{type:OperatorId,search operators} the field
// value is considered as matching the <code>Criterion</code> if <i>any</i> of the field values
// match the Criterion.  Specifically, this is true of all operators that have an
// +link{type:OperatorValueType,operatorValueType} of "fieldType" or "valueRange".
// <p>
// For operators that compare against other fields in same record, such as "equalsField",
// if the other field is <i>not</i> <code>multiple:true</code>, matching works the same as for
// normal operators, that is, as if <code>criterion.value</code> directly contained the value
// rather than the name of another field.
// <p>
// If the other field is <i>also</i> multiple:true, only "equalsField", "notEqualsField",
// "iEqualsField" and "iNotEqualsField" are allowed (any other <code>operator</code> will
// cause a warning and be ignored) and the set of values in the field must be identical (aside
// from case, for operators prefixed with "i") and in identical order to match.
// <p>
// For the <code>inSet</code> operator, the field matches if there is any intersection between
// the field values and the array of values provided in <code>criterion.value</code>.
// <code>notInSet</code> is the reverse.
// <p>
// Finally, for "isNull" and "isNotNull", an empty Array is considered non-null.  For example,
// if you use dataFormat:"json" and the field value is provided to the browser as
// <code>[]</code> (JSON for an empty Array), the field is considered non-null.
// <P>
// <h4>XML Serialization</h4>
// <p>
// Specifically for XML serialization and deserialization, <code>multiple:true</code> behaves
// similarly to the 
// +externalLink{http://www.google.com/search?hl=en&q=soap+array,SOAP array idiom}, that is,
// there will be a "wrapper element" named after the field name, whose contents will be several
// elements of the specified +link{dataSourceField.type,field.type}.  
// <P>
// For example, +link{layout.members} is declared with <code>type:"Canvas",
// multiple:true</code>.  The correct XML format is thus:
// <pre>
// &lt;VLayout&gt;
//     &lt;members&gt;
//         &lt;Canvas ID="myCanvas" ... /&gt;
//         &lt;ListGrid ID="myGrid" .../&gt;
//         &lt;Toolstrip ID="myToolStrip" ... /&gt;
//     &lt;/members&gt;
// &lt;/VLayout&gt;
// </pre>
// <P>
// See +link{dataSourceField.childTagName} for customizing the tagName used for subelements.
//
// @group xmlSerialize
// @group componentSchema
// @serverDS allowed
// @visibility external
//<

//> @attr dataSourceField.multipleValueSeparator (String : ", " : IR)
// For fields that are +link{dataSourceField.multiple,multiple:true}, the separator used 
// between values when they are displayed.
// @visibility external
//<

//> @attr dataSourceField.childTagName (String : null : IR)
// For a field that is +link{multiple,multiple:"true"}, controls the name of the XML tag used for each
// subelement during +link{dataSource.xmlSerialize()}.
// <P>
// If unset, the default tag name is "value" for a field of simple type, and for a field of
// DataSource type, is the tagName or ID of the DataSource (as though
// <code>xmlSerialize()</code> were called on the child DataSource).
//
// @group xmlSerialize
// @group componentSchema
// @serverDS allowed
// @visibility external
//<

//> @attr dataSourceField.propertiesOnly (boolean : null : IR)
// For use in +link{group:componentSchema} for fields that contain other components, this flag
// suppresses auto-construction for subcomponents that appear under this field.
// <P>
// For example, the +link{VLayout} schema sets this for its +link{Layout.members,members}
// property, so that when a VLayout is constructed via XML as follows:
// <pre>
// &lt;VLayout&gt;
//     &lt;members&gt;
//         &lt;ListGrid ID="myGrid" .../&gt;
//         &lt;Toolstrip ID="myToolStrip" ... /&gt;
//     &lt;/members&gt;
// &lt;/VLayout&gt;
// </pre>
// The ListGrid and ToolStrip do not construct themselves automatically.  Instead, the VLayout
// receives the properties of the ListGrid and ToolStrip as ordinary JavaScript Objects, with
// the special property <code>_constructor</code> set to the name of the class that should be
// constructed.
//
// @group componentSchema
// @serverDS allowed
// @visibility external
//<

//> @attr dataSourceField.inapplicable (boolean : null : IR)
// For use in +link{group:componentSchema}, a field +link{dataSource.inheritsFrom,inherited} from
// another schema can be redeclared with this property set in order to indicate that the
// property should not be used.
// <P>
// This is primarily used to influence +link{group:visualBuilder}.  For simple type properties,
// this avoids the property appearing in the Component Editor.  
// <P>
// For fields that hold subcomponents, this prevents inappropriate drag and drop.  For example,
// a custom class called <code>MyDialog</code> may automatically create a series of children,
// and not allow arbitrary other children to be added.  In this case, the inherited property
// +link{Canvas.children} should be marked inapplicable in order to prevent arbitrary
// components being dropped onto a <code>MyDialog</code> instance.
//
// @group componentSchema
// @serverDS allowed
// @visibility external
//<

//> @attr dataSourceField.group (string : null : IR)
// For use in +link{group:componentSchema}, indicates what group to place the property in when
// editing in Visual Builder.
//
// @group componentSchema
// @serverDS allowed
// @visibility external
//<

// Javabean/POJO Binding
// ---------------------------------------------------------------------------------------

//> @attr dataSourceField.javaClass (String : null : IR)
// For use with the SmartClient server when populating Java Beans / POJOs based on data
// contained in a DSRequest, <code>javaClass</code> specifies the fully qualified Java
// className to be created and passed to the setter for the Java Bean Property with the same
// name as this field.   <code>javaClass</code> is used both when manually calling
// DataSource.setProperties() and when auto-populating POJO arguments of a +link{dmiOverview,DMI} method.
// <P>
// The Java class to create does not normally have to be specified: SmartClient will
// use Java reflection to inspect the type of argument expected by a setter method and will
// attempt conversion of inbound data to that type.  As described in the documentation for
// DataTools.setProperties(), this works for almost all typical cases.  However
// <code>field.javaClass</code> is useful for:
// <ul>
// <li> subobject of abstract or interface type: in this case Java Reflection is not sufficient
// to discover the concrete type that should be instantiated, and <code>javaClass</code> should be
// set instead.</li>
// <li> subobject of Collection or Map type, when Java generics are not used or the Collection
// member type or Map value type is abstract.  When Java generics are used (for example the
// setter takes an argument is of type Collection&lt;SomePOJO&gt; or Map&lt;KeyType,SomePOJO&gt;,
// SmartClient will automatically attempt to convert inbound data to the type of the members of
// the Collection or values of the Map.  Without generics, <code>javaClass</code> needs to be
// specified.  Note that <code>javaClass</code> will take precedence over generics if both 
// are used. Also note that +link{dataSourceField.javaCollectionClass} can be specified if a
// particular Collection or Map type is needed, and +link{dataSourceField.javaKeyClass} can be
// specified for a field of type <code>java.util.Map</code>.</li>
// </ul>
// <b>NOTE:</b> It is also possible to make SmartClient Server determine the javaClass to use 
// dynamically at runtime. See the description of <code>DataSource.getPropertyJavaClass()</code> 
// in the server documentation for details.
//
// @serverDS only
// @visibility external
//<

//> @attr dataSourceField.javaCollectionClass (String : null : IR)
// See +link{dataSourceField.javaClass} - when auto-populating of Java Beans / POJOs based on 
// inbound DSRequest data, for a field of type Collection or Map, 
// <code>javaCollectionClass</code> can 
// be used to specify a particular concrete class to use.  If not specified, and a concrete
// Collection or Map class cannot be discovered using Java Reflection, the following concrete
// classes are used:
// <ul>
// <li> <code>java.util.ArrayList</code> is used for fields of type <code>List</code>
// <li> <code>java.util.HashSet</code> is used for fields of type <code>Set</code>
// <li> <code>java.util.LinkedList</code> is used for fields of type <code>Queue</code>
// <li> <code>org.apache.commons.collections.map.LinkedMap</code> is used for fields of type <code>Map</code>
// <li> <code>java.util.ArrayList</code> is used for fields that are otherwise of type <code>Collection</code>
// </ul>
// Note that this value is used even if the target Collection or Map is declared as a concrete
// class.  So, for example, if you set <code>javaCollectionClass</code> to 
// <code>java.util.LinkedList</code> but your setter method accepts a 
// <code>java.util.ArrayList</code>, you will get a ClassCastException.
//
// @serverDS only
// @visibility external
//<

//> @attr dataSourceField.javaKeyClass (String : null : IR)
// See +link{dataSourceField.javaClass} - when auto-populating of Java Beans / POJOs based on 
// inbound DSRequest data, for a field of Map type, <code>javaKeyClass</code> can be used to
// specify a particular concrete class for the map keys.  If not specified, and a concrete
// type cannot be discovered using Java Reflection, <code>java.lang.Object</code> is used.
//
// Note that <code>javaKeyClass</code> take precedence over generics if both are used.
//
// @serverDS only
// @visibility external
//<


// Exporting
// ---------------------------------------------------------------------------------------

//> @attr dataSourceField.exportTitle (String : null : IR)
//	Optional different field-title used for exports.
//
// @serverDS allowed
// @visibility external
//<

//> @attr dataSourceField.canExport (Boolean : null : IR)
// Dictates whether the data in this field be exported.  Explicitly setting 
// <i>canExport</i> to false overrides the setting on any component-fields, such
// as +link{listGridField.canExport, ListGrid fields}.
//
// @serverDS allowed
// @visibility external
//<

// Upload
// --------------------------------------------------------------------------------------------

//> @attr dataSourceField.uploadFieldName (String : null : IR)
// Used by the +link{BatchUploader} to map a field in an upload file to this 
// dataSourceField.  This is only necessary if the dataSourceField's name and title differ 
// from the name of the field in the upload file (SmartClient will automatically map upload 
// fields using the dataSourceField's title, if possible, if it does not get a direct match 
// on field name).
//
// @serverDS allowed
// @visibility batchUploader
//<

//> @attr dataSourceField.showFileInline    (boolean : null : [IR])
// For a field of type:"imageFile", indicates whether to stream the image and display it
// inline or to display the View and Download icons.
// 
// @serverDS allowed
// @visibility external
//<

//> @attr dataSourceField.imageWidth (number or String: null : [IR])
// Width of the image-content of this field.  If set as a string, represents the name of 
// another field in the record that holds the imageWidth.  Applicable only to fields of image 
// type or fields that use a +link{class:ViewFileItem, ViewFileItem} as an editor.
// @serverDS allowed
// @visibility external
//<

//> @attr dataSourceField.imageHeight (number or String: null : [IR])
// Height of the image-content of this field.  If set as a string, represents the name of 
// another field in the record that holds the imageHeight.  Applicable only to fields of image 
// type or fields that use a +link{class:ViewFileItem, ViewFileItem} as an editor.
// @serverDS allowed
// @visibility external
//<

//> @attr dataSourceField.imageSize (number or String: null : [IR])
// Width and height of the image-content of this field.  If set as a string, represents the 
// name of another field in the record that holds the imageSize.  Applicable only to fields 
// of image type or fields that use a +link{class:ViewFileItem, ViewFileItem} as an editor.
// @serverDS allowed
// @visibility external
//<


// Role-based security
// ----------------------------------------------------------------------------------------

//> @groupDef fieldLevelAuth
// Properties that provide declarative security features at the level of individual fields
// @title Field-Level Security
// @visibility external
//< 

//> @attr dataSourceField.viewRequiresAuthentication (boolean : null : IR)
// Whether a user must be authenticated in order to fetch this field.  If this property is 
// set and the user is not authenticated, SmartClient Server will not return values for this
// field to the client.
// <p>
// The way this behavior works is to remove the field from the expressed or implied list of
// fields to return.  A list of fields can be expressed using +link{OperationBinding.outputs}
// or +link{dsRequest.outputs}; in the absence of those, the entire set of fields defined in 
// the DataSource is implied.  You can override this behavior on a per-request basis by 
// providing a DMI that just re-adds the names of the fields you want for that particular 
// request to the "consolidated outputs".  See the server-side Javadoc for 
// <code>DSRequest.getConsolidatedOutputs()</code>.
// <p>
// In addition to this behavior of trimming off values in fetched records, a field that fails
// a field-level security check (either this one or one of the two more specific checks, 
// +link{viewRequiresRole,viewRequiresRole} and +link{viewRequires,viewRequires}) will be 
// altered in the version of the dataSource returned to the client by the 
// +link{group:loadDSTag,&lt;loadDS&gt; JSP tag} or 
// +link{group:dataSourceDeclaration,DataSourceLoader servlet}.
// <ul>
// <li>If the field is also not editable (see the discussion of this below), it will be marked 
// +link{canView,canView}:false, and thus effectively be inert and unusable on the client</li>
// <li>If the field is editable, it will be marked +link{hidden,hidden}.  By default, this means
// that +link{dataBoundComponent}s will not show the field, though this can be overridden for 
// individual components.  If you do choose to unhide the field, bear in mind that it will 
// never actually show any data, because the underlying security test has still failed</li>
// </ul>
// By default, view constraints automatically imply edit constraints - almost all of the time,
// if a user is not allowed to view a field, he should not be allowed to update it either.  If
// you specify a view constraint, the system behaves as if you had also specified the 
// corresponding edit constraint - so <code>viewRequiresAuthentication</code> on a field 
// implies <code>+link{dataSourceField.editRequiresAuthentication,editRequiresAuthentication}</code>.
// You override this behavior simply by specifying some kind of edit constraint for the field,
// either a different one (eg, "users" can view a field, but only "managers" can update it) or 
// a blank one (if you have some unusual requirement whereby users are not allowed to view a 
// field, but they can update it).
//
// @requiresModules SCServer
// @see dataSourceField.editRequiresAuthentication
// @group fieldLevelAuth
// @serverDS only
// @visibility external
//<

//> @attr dataSourceField.editRequiresAuthentication (boolean : null : IR)
// Whether a user must be authenticated in order to write to this field.  If this property is 
// set and the user is not authenticated, the SmartClient Server will not allow this field to
// be updated or initialized.
// <p>
// This property affects two things.  Firstly, the server determines when the DataSource is 
// first loaded if we have an authenticated user; if we don't, the field is marked 
// <code>canEdit: false</code>.  Secondly, when an insert or update request is received from 
// the client, the server removes the field from the <code>values</code> clause before the
// request is processed.
// <p>
// Note that you can override this behavior.  The <code>canEdit</code> setting can be overridden 
// on the the client-side DataSource like any other client-side property.  The value-removing 
// behavior can be overridden on a per-request basis by providing a DMI that re-adds values for
// the fields you want for that particular request to the values object (the values originally 
// sent up by the client are available on the DSRequest).  See the server-side Javadocs for 
// <code>DSRequest.getValues()</code> and <code>DSRequest.getClientSuppliedValues()</code>.
//
// @requiresModules SCServer
// @see dataSourceField.viewRequiresAuthentication
// @group fieldLevelAuth
// @serverDS only
// @visibility external
//<

//> @attr dataSourceField.initRequiresAuthentication (boolean : null : IR)
// Whether a user must be authenticated in order to initialize to this field.  If this property is 
// set and the user is not authenticated, the SmartClient Server will not allow this field to
// be initialized.
//
// @requiresModules SCServer
// @see dataSourceField.editRequiresAuthentication
// @group fieldLevelAuth
// @serverDS only
// @visibility external
//<

//> @attr dataSourceField.updateRequiresAuthentication (boolean : null : IR)
// Whether a user must be authenticated in order to update to this field.  If this property is 
// set and the user is not authenticated, the SmartClient Server will not allow this field to
// be updated.
//
// @requiresModules SCServer
// @see dataSourceField.editRequiresAuthentication
// @group fieldLevelAuth
// @serverDS only
// @visibility external
//<

//> @attr dataSourceField.viewRequiresRole (String : null : IR)
// Comma-separated list of user roles that are allowed to fetch this field. If the current user
// has any of the roles listed, values for the field will be fetched.  Please see 
// +link{operationBinding.requiresRole} for further details of SmartClient's declarative 
// role-based security.  Please also see 
// +link{dataSourceField.fetchRequiresAuthentication,fetchRequiresAuthentication} for details
// of how declarative field-level security settings can be overridden per-request.
//
// @requiresModules SCServer
// @see dataSourceField.viewRequiresAuthentication
// @see dataSourceField.editRequiresRole
// @group fieldLevelAuth
// @serverDS only
// @visibility external
//<

//> @attr dataSourceField.editRequiresRole (String : null : IR)
// Comma-separated list of user roles that are allowed to write to this field. If the current
// user does not have any of the roles listed, the system will not allow this field to be 
// initialized or updated.  Please see +link{operationBinding.requiresRole} for further 
// details of SmartClient's declarative role-based security.  Please also see 
// +link{dataSourceField.fetchRequiresAuthentication,editRequiresAuthentication} for details
// of how declarative field-level security settings can be overridden per-request.
// <p>
// <b>NOTE:</b> This property prevents both initialization and updates for a field.  If you 
// have a need to prevent <em>just</em> initialization or <em>just</em> updates, you can use
// +link{dataSourceField.initRequiresRole} or +link{dataSourceField.updateRequiresRole}.
//
// @requiresModules SCServer
// @see dataSourceField.editRequiresAuthentication
// @see dataSourceField.viewRequiresRole
// @see dataSourceField.initRequiresRole
// @see dataSourceField.updateRequiresRole
// @group fieldLevelAuth
// @serverDS only
// @visibility external
//<

//> @attr dataSourceField.initRequiresRole (String : null : IR)
// Comma-separated list of user roles that are allowed to initialize this field. If the current
// user does not have any of the roles listed, the system will not allow this field to be 
// initialized.
// <p>
// <b>NOTE:</b> This property only prevents initialization of a field; updates will still be 
// allowed.  You should only use this property if you have a special requirement; ordinarily,
// use +link{dataSourceField.editRequiresRole}, which applies security for both types of write.
//
// @requiresModules SCServer
// @see dataSourceField.editRequiresAuthentication
// @see dataSourceField.editRequiresRole
// @see dataSourceField.updateRequiresRole
// @see dataSourceField.initRequires
// @group fieldLevelAuth
// @serverDS only
// @visibility external
//<

//> @attr dataSourceField.updateRequiresRole (String : null : IR)
// Comma-separated list of user roles that are allowed to update this field. If the current
// user does not have any of the roles listed, the system will not allow this field to be 
// updated.
// <p>
// <b>NOTE:</b> This property only prevents a field from being updated; initialization will 
// still be allowed.  You should only use this property if you have a special requirement; 
// ordinarily, use +link{dataSourceField.editRequiresRole}, which applies security for both 
// types of write.
//
// @requiresModules SCServer
// @see dataSourceField.editRequiresAuthentication
// @see dataSourceField.editRequiresRole
// @see dataSourceField.initRequires
// @group fieldLevelAuth
// @serverDS only
// @visibility external
//<

//> @attr dataSourceField.viewRequires (VelocityExpression : null : IR)
// Indicates that the specified +link{group:VelocityExpression} must evaluate to true if
// values for the field are to be fetched.  If the specified expression does not evaluate
// to true, the field will be dropped as described for 
// +link{dataSourceField.fetchRequiresAuthentication,viewRequiresAuthentication}.
// <p>
// In addition to the normal context variables available to Velocity expressions in SmartClient,
// expressions you write for field-level <code>requires</code> clauses - <code>viewRequires</code>, 
// +link{editRequires,editRequires}, +link{initRequires,initRequires} and 
// +link{updateRequires,updateRequires} - can reference two additional variables: 
// <code>$fieldName</code> and <code>$dsName</code>.  These are the names of the dataSource and 
// field currently undergoing <code>requires</code> checks.  They are helpful because they allow
// you to write a generic checker function that can be used to handle <code>requires</code>
// checks for multiple fields and dataSources.
//
// @requiresModules SCServer
// @see dataSourceField.viewRequiresAuthentication
// @see dataSourceField.viewRequiresRole
// @see dataSourceField.editRequires
// @group fieldLevelAuth
// @serverDS only
// @visibility external
//<

//> @attr dataSourceField.editRequires (VelocityExpression : null : IR)
// Indicates that the specified +link{group:VelocityExpression} must evaluate to true if
// writes to this field are to be allowed.  If the specified expression does not evaluate
// to true, the field will be removed from the request as described for 
// +link{dataSourceField.editRequiresAuthentication,editRequiresAuthentication}.
// <p>
// In addition to the normal context variables available to Velocity expressions in SmartClient,
// expressions you write for field-level <code>requires</code> clauses - <code>editRequires</code>, 
// +link{viewRequires,viewRequires}, +link{initRequires,initRequires} and 
// +link{updateRequires,updateRequires} - can reference two additional variables: 
// <code>$fieldName</code> and <code>$dsName</code>.  These are the names of the dataSource and 
// field currently undergoing <code>requires</code> checks.  They are helpful because they allow
// you to write a generic checker function that can be used to handle <code>requires</code>
// checks for multiple fields and dataSources.
// <p>
// <b>NOTE:</b> This property prevents both initialization and updates for a field.  If you 
// have a need to prevent <em>just</em> initialization or <em>just</em> updates, you can use
// +link{dataSourceField.initRequires} or +link{dataSourceField.updateRequires}.
//
// @requiresModules SCServer
// @see dataSourceField.editRequiresAuthentication
// @see dataSourceField.editRequiresRole
// @see dataSourceField.viewRequires
// @group fieldLevelAuth
// @serverDS only
// @visibility external
//<

//> @attr dataSourceField.initRequires (VelocityExpression : null : IR)
// Indicates that the specified +link{group:VelocityExpression} must evaluate to true if
// initializations of this field are to be allowed.  If the specified expression does not 
// evaluate to true, the field will be removed from the request as described for 
// +link{dataSourceField.editRequiresAuthentication,editRequiresAuthentication}.
// <p>
// In addition to the normal context variables available to Velocity expressions in SmartClient,
// expressions you write for field-level <code>requires</code> clauses - <code>initRequires</code>, 
// +link{viewRequires,viewRequires}, +link{editRequires,editRequires} and 
// +link{updateRequires,updateRequires} - can reference two additional variables: 
// <code>$fieldName</code> and <code>$dsName</code>.  These are the names of the dataSource and 
// field currently undergoing <code>requires</code> checks.  They are helpful because they allow
// you to write a generic checker function that can be used to handle <code>requires</code>
// checks for multiple fields and dataSources.
// <p>
// <b>NOTE:</b> This property only prevents initialization of a field; updates will still be 
// allowed.  You should only use this property if you have a special requirement; ordinarily,
// use +link{dataSourceField.editRequires}, which applies security for both types of write.
//
// @requiresModules SCServer
// @see dataSourceField.editRequiresAuthentication
// @see dataSourceField.editRequiresRole
// @see dataSourceField.viewRequires
// @group fieldLevelAuth
// @serverDS only
// @visibility external
//<

//> @attr dataSourceField.updateRequires (VelocityExpression : null : IR)
// Indicates that the specified +link{group:VelocityExpression} must evaluate to true if
// updates to this field are to be allowed.  If the specified expression does not 
// evaluate to true, the field will be removed from the request as described for 
// +link{dataSourceField.editRequiresAuthentication,editRequiresAuthentication}.
// <p>
// In addition to the normal context variables available to Velocity expressions in SmartClient,
// expressions you write for field-level <code>requires</code> clauses - <code>updateRequires</code>, 
// +link{viewRequires,viewRequires}, +link{editRequires,editRequires} and 
// +link{initRequires,initRequires} - can reference two additional variables: 
// <code>$fieldName</code> and <code>$dsName</code>.  These are the names of the dataSource and 
// field currently undergoing <code>requires</code> checks.  They are helpful because they allow
// you to write a generic checker function that can be used to handle <code>requires</code>
// checks for multiple fields and dataSources.
// <p>
// <b>NOTE:</b> This property only prevents update of a field; initialization will still be 
// allowed.  You should only use this property if you have a special requirement; ordinarily,
// use +link{dataSourceField.editRequires}, which applies security for both types of write.
//
// @requiresModules SCServer
// @see dataSourceField.editRequiresAuthentication
// @see dataSourceField.editRequiresRole
// @see dataSourceField.viewRequires
// @group fieldLevelAuth
// @serverDS only
// @visibility external
//<

//> @attr dataSourceField.creatorOverrides (boolean : null : IR)
// Indicates that normal declarative security rules specified on this field are waived for 
// rows that were created by the current user, as described in the discussion of 
// +link{DataSource.creatorOverrides,dataSource.creatorOverrides}.  This setting overrides 
// <code>dataSource.creatorOverrides</code>, for this field only.
// <p>
// Note that field-level <code>creatorOverrides</code> is conceptually slightly different to
// the setting at DataSource and OperationBinding levels.  To give the example of a fetch 
// operation, at the other two levels, it results in a limited set of rows being returned, 
// rather than a security exception being thrown.  At the field-level, it results in values 
// for individual fields being present in those rows, rather than them being stripped out on
// the server.
//
// @requiresModules SCServer
// @see dataSourceField.editRequires
// @see dataSourceField.viewRequires
// @see dataSource.creatorOverrides
// @group fieldLevelAuth
// @serverDS only
// @visibility external
//<


// Auditing
// ---------------------------------------------------------------------------------------------

//> @attr dataSource.audit (boolean : false : [IR])
// Enables saving of a log of changes to this DataSource in a second DataSource with the same
// fields, called the "audit DataSource".  <b>NOTE:</b>: this feature applies to Enterprise
// Edition only; for more information on edition-specific features, see
// +externalLink{http://smartclient.com/product}.
// <p>
// When auditing is enabled, every time a DSRequest modifies this DataSource, a Record is added
// to the audit DataSource that shows the record as it existed after the change was made (or
// for a "remove", the values of the record at the time of deletion).  In addition, the audit
// DataSource has the following additional metadata fields:
// <ul>
// <li> +link{auditTypeFieldName,"audit_operationType"}: type of the change ("update", "add" or "remove")
// <li> +link{auditUserFieldName,"audit_modifier"}: username of the user that made the change.  The username is
//      determined in the same way that the 
//      +link{operationBinding.requiresRole,Declarative Security} subsystem determines the 
//      current user.
// <li> +link{auditTimeStampFieldName,"audit_changeTime"}: a field of type "datetime" recording the timestamp of
//      the change
// <li> +link{auditRevisionFieldName,"audit_revision"}: a field of type "sequence" recording a simple
//      increasing integer value
// </ul>
// <p>
// If any of the field names above collide with field names of the DataSource being audited,
// an integer suffix will also be added, starting with 2 (for example, "audit_modifier2", then
// "audit_modifier3", etc).
// <p>
// To omit a data field from the automatically generated audit DataSource, just set
// +link{dataSourceField.audit} to false.
// <p>
// If auditing is enabled for a SQLDataSource, the SQL table necessary to store audit
// information will also be automatically created, such that <code>audit="true"</code> is the
// sole setting required to enable the feature and cause actual storage of a change log.
// For other types of DataSources, table creation (or creation of other kinds of underlying
// storage) must be done manually.
// <p>
// Unless otherwise specified with +link{dataSource.auditDataSourceID}, the name of the audit 
// DataSource will be <code>audit_[OriginalDSID]</code>.  Automatically created audit
// DataSources can be loaded and queried just like other DataSources, using the
// DataSourceLoader, and using the server-side API
// <code>DataSource.getAuditDataSource()</code>.  However, you <b>must</b> load the DataSource
// being audited before loading its automatically created audit DataSource.
// <p>
// The audit DataSource can also be manually created.  In this case, you can 
// can either follow the naming conventions described above for the ID of the audit DataSource
// and the names of metadata fields, or use the linked properties to assign custom names.  If
// you omit any data fields from the tracked DataSource in your audit DataSource, those fields
// will be ignored for auditing purposes, exactly as though +link{dataSourceField.audit} had
// been set to false for an automatically-generated audit DataSource.
//
// @group serverDataIntegration
// @requiresModules SCServer
// @example auditing
// @serverDS only
// @visibility auditing
//<

//> @attr dataSourceField.audit (boolean : true : [IR])
// Setting <code>audit</code> to <code>false</code> explicitly indicates that this field will
// not be saved to the audit DataSource when +link{dataSource.audit,auditing} is enabled.
//
// @serverDS only
// @visibility auditing
//<

//> @attr dataSource.auditDataSourceID (string : null : [IR])
// For DataSources with +link{audit,auditing enabled}, optionally specifies the ID of the audit
// DataSource.  If this property is not specified, the ID of the audit DataSource will
// be <code>audit_[OriginalDSID] </code>
//
// @serverDS only
// @visibility auditing
//<

//> @attr dataSource.auditUserFieldName (string : "audit_modifier" : [IR])
// For DataSources with +link{audit,auditing enabled}, specifies the field name used to store
// the user who performed the operation.  If empty string is specified as the field name, the
// audit DataSource will not store this field.
//
// @serverDS only
// @visibility auditing
//<

//> @attr dataSource.auditTimeStampFieldName (string : "audit_changeTime" : [IR])
// For DataSources with +link{audit,auditing enabled}, specifies the field name used to store
// the timestamp when the operation was performed (in a field of type "datetime").  If empty
// string is specified as the field name, the audit DataSource will not store this field.
//
// @serverDS only
// @visibility auditing
//<

//> @attr dataSource.auditRevisionFieldName (string : "audit_revision" : [IR])
// For DataSources with +link{audit,auditing enabled}, specifies the field name used to store
// the revision number for the change (in a field of type "sequence").  If empty
// string is specified as the field name, the audit DataSource will not store this field.
//
// @serverDS only
// @visibility auditing
//<

//> @attr dataSource.auditTypeFieldName (string : "audit_operationType" : [IR])
// For DataSources with +link{audit,auditing enabled}, specifies the field name used to store
// the +link{type:DSOperationType,operationType} (in a field of type "text").  If empty
// string is specified as the field name, the audit DataSource will not store this field.
//
// @serverDS only
// @visibility auditing
//<


// Miscellaneous
// --------------------------------------------------------------------------------------------

//> @attr dataSourceField.customSQL (boolean : null : IR)
// For a DataSource with +link{dataSource.serverType,serverType} "sql" or "hibernate",
// indicates that this field should be omitted by default from all SQL or Hibernate operations,
// and will only be used with +link{group:customQueries,custom queries}.
// <P>
// Having marked a field as <code>customSQL</code> you can refer to it via
// $criteria.<i>fieldName</i> or $values.<i>fieldName</i> in customized queries.
// <P>
// The following are situations where you would <b>not</b> use <code>customSQL</code>:
// <ul>
// <li>simple joins where you want to enable users to see and search on a field from another
// table; consider +link{dataSourceField.includeFrom} instead
// <li>fields where you want to calculate or transform values in SQL on load or save, but
// always perform the same calculation for each operationType; consider instead
// +link{sqlStorageStrategy} for some common cases, or 
// +link{customSelectExpression}, +link{customUpdateExpression} and
// +link{customInsertExpression} for full customization
// <li>a special fetch is needed where the field needs to be excluded from the
// $defaultWhereClause so that it can be used in a custom &lt;whereClause&gt; - consider
// +link{operationBinding.excludeCriteriaFields} instead
// </ul>
// <P>
// Use customSQL in situations like:
// <ul>
// <li>there are multiple variations of the "fetch" operation with different
// +link{operationBinding.operationId,operationIds}, and the field is only used in some of them;
// in that case, consider using +link{operationBinding.customFields} to selectively re-introduce
// SQL generation for the field only in operations where it's used.
// <li>the field represents hidden criteria on a field in another table where the field is
// never shown to the user
// <li>the field is a write-only value only saved in some operations
// <li>more than one data access strategy is in use (eg direct SQL for fetch and bean-based
// persistence accessed via DMI for saves) and certain fields are not available in SQL
// </ul>
//
// @serverDS allowed
// @visibility customSQL
//<


//>
// NOTE: This used to be customSQLExpression; we renamed it to customSelectExpression when we
// introduced customUpdateExpression and customInsertExpression, to avoid ambiguity.  The 
// system still supports customSQLExpression, for backompat
//<

//> @attr dataSourceField.customSelectExpression (string : null : IR)
// This property indicates that this field represents a custom expression that should be 
// embedded in the generated SQL instead of a reference to this field.  For example, if 
// you have a field <code>partialName</code> where this value is set to 
// <code>SUBSTR(surname, 2)</code>, the generated SQL would look similar to this:
// <pre>
//   SELECT ... SUBSTR(surname, 2) AS partialName ...
// </pre>
// Note that adding "AS partialName" as shown above is required - this is how the framework
// identifies the expression as producing a value for this particular DataSourceField when
// results are received from the database.
// <p>
// Fields with <code>customSelectExpression</code> set can be used for sorting and filtering in
// the normal way, but they are only applicable to update-type operations if you also provide a
// corresponding +link{customUpdateExpression,customUpdateExpression} and/or 
// +link{customInsertExpression,customInsertExpression}.  See the documentation 
// for those methods for the rules of how they are applied.
// <p>
// You can use Velocity expressions in your <code>customSelectExpression</code>s, and the 
// +link{group:velocitySupport,usual context variables} are available.  Note that the results
// of Velocity evaluations are automatically quoted and escaped by default; this behavior can
// be switched off - see +link{autoQuoteCustomExpressions,autoQuoteCustomExpressions}, and 
// particularly note the warnings about its use.
// <p>
// Note that this property only applies to users of the SmartClient server using dataSources of
// type "sql".
//
// @see customUpdateExpression
// @see customInsertExpression
// @see customCriteriaExpression
// @see autoQuoteCustomExpressions
// @serverDS only
// @visibility customSQL
//<

//> @attr dataSourceField.customUpdateExpression (string : null : IR)
// This property specifies the value to use to update this column on "update" operations.
// The value of this property will be passed through Velocity evaluation and then embedded
// directly in the SQL generated to perform the update.  It can be used in conjunction with
// +link{customSelectExpression,customSelectExpression} to provide bi-directional mapping 
// between application data formats and persisted data formats.  Or, it can be used 
// unilaterally as a means of silently enforcing data rules - for example, ensuring that all 
// values for a given field are in upper case.
// <P>
// You can use Velocity expressions in your <code>customUpdateExpression</code>s, and the 
// +link{group:velocitySupport,usual context variables} are available.  Note that the results
// of Velocity evaluations are automatically quoted and escaped by default; this behavior can
// be switched off - see +link{autoQuoteCustomExpressions,autoQuoteCustomExpressions}, and 
// particularly note the warnings about its use.
// <p>
// The value must be a string that will end up as a valid SQL snippet after Velocity
// evaluation.  The following examples are valid:
// <ul>
// <li><code>"'Hello world'"</code></li>
// <li><code>"$values.someTextField.toUpperCase()"</code></li>
// <li><code>"SUBSTR(ANOTHER_COLUMN, 1, 10)"</code></li>
// <li><code>"SUBSTR($values.someTextField.toUpperCase(), 1, 10)"</code></li>
// <li><code>"SOME_NUMERIC_COLUMN * 1000"</code></li>
// <li><code>"$values.someDateField.getMonth() + 1"</code></li>
// </ul>
// The first example is a literal and so it must be enclosed in quotes to render valid SQL.
// The second example is also a literal, but as it is the result of a Velocity evaluation, it
// will be automatically quoted as appropriate.  The third example is an actual SQL snippet 
// and does not need to be enclosed in quotes (indeed, would fail if it were).  The fourth 
// example shows an SQL snippet containing a literal - again, because the literal is the result
// of a Velocity evaluation, it will be automatically quoted.  The remaining two examples both
// yield literals, but as they are numeric literals they must not be quoted.
// <P>
// When an "update" operation runs, any fields that specify 
// +link{customSelectExpression,customSelectExpression} but do <b>not</b> specify 
// customUpdateExpression will be ignored.  When an "add" operation runs, this property acts 
// as a default for any fields that do not specify a 
// +link{customInsertExpression,customInsertExpression}; similar to update, any fields that 
// specify a customSelectExpression but do not specify either a customUpdateExpression or 
// customInsertExpression, will be ignored when "add" operations run.
// <p>
// Note that this property only applies to users of the SmartClient server using dataSources of
// type "sql".
//
// @see customSelectExpression
// @see customInsertExpression
// @see customCriteriaExpression
// @see autoQuoteCustomExpressions
// @serverDS only
// @visibility customSQL
//<

//> @attr dataSourceField.customInsertExpression (string : null : IR)
// This property is similar to +link{customUpdateExpression,customUpdateExpression}; its value
// is used during "add" operations.  If you do not specify this property, but do specify a 
// <code>customUpdateExpression</code>, the <code>customUpdateExpression</code> is used instead.
// <P>
// Everything that applies to <code>customUpdateExpression</code> also applies to this property,
// including the observation that fields which specify a 
// +link{customSelectExpression,customSelectExpression} but no corresponding 
// <code>customUpdateExpression</code> or <code>customInsertExpression</code>, will be ignored
// when adding new records.
// <p>
// Note that this property only applies to users of the SmartClient server using dataSources of
// type "sql".
//
// @see customUpdateExpression
// @see customSelectExpression
// @see customCriteriaExpression
// @see autoQuoteCustomExpressions
// @serverDS only
// @visibility customSQL
//<

//> @attr dataSourceField.customCriteriaExpression (string : null : IR)
// This property indicates that this field should be represented by a custom expression
// embedded in the WHERE clause of the generated SQL, instead of the generated expression
// containing this field name that would ordinarily be used.  You use this property when 
// you have to accomodate some special way of using a field's value in criteria, other than by
// directly comparing it to the criteria.  For example, if you have a column that contains 
// bit-packed information you will generally need to perform a bitwise AND to filter on that 
// column, rather than an equality check.  In this case, you would use a 
// <code>customCriteriaExpression</code> something like this (in Oracle):
// <p>
// <code>&nbsp;&nbsp;&nbsp;BITAND(myField, $criteria.myField)+0 = $criteria.myField</code>
// <p>
// Or this (in MySQL or SQL Server):
// <p>
// <code>&nbsp;&nbsp;&nbsp;myField & $criteria.myField = $criteria.myField</code>
// <p>
// As this example shows, a <code>customCriteriaExpression</code> is expected to be a complete
// logical expression that provides both sides of the comparison.  After Velocity processing,
// this example would result in SQL similar to this (for the MySQL case); the colored part 
// comes entirely from the <code>customCriteriaExpression</code>:
// <p>
// <code>SELECT myField, myOtherField FROM myTable WHERE <font color='blue'>myField & 32 = 32</font></code>
// <p>
// <code>customCriteriaExpression</code> can also be used with <code>AdvancedCriteria</code>.
// Note that the point mentioned above, about a <code>customCriteriaExpression</code> being a 
// complete logical expression that provides both sides of the comparison still applies.  This
// means that when you use <code>customCriteriaExpression</code> in conjunction with 
// <code>AdvancedCriteria</code>, you effectively switch off support for different 
// +link{type:OperatorId}s to be used on that field, unless you use Velocity expressions 
// containing the special <code>$criteriaOperator</code> variable (see below).
// <p>
// When you use <code>customCriteriaExpression</code> with <code>AdvancedCriteria</code>, the 
// way you access criteria values differs slightly.  One option is to use the 
// <code>$advancedCriteria</code> Velocity variable, as described in the "Using AdvancedCriteria"
// section of the +link{group:customQuerying,custom querying overview}.  However, this 
// variable only returns the value from the first +link{object:Criterion} that uses the field,
// as found by depth-first search.  If your <code>AdvancedCriteria</code> contains multiple 
// references to this field, this will not be satisfactory.  Therefore, specifically for 
// <code>customCriteriaExpression</code>, we provide another Velocity variable, 
// <code>$criteriaValue</code>.  This variable means "the value on a Criterion referencing 
// this field".  An example may serve to clarify this:
// <p>
// Assume you need to extend the above example to filter records where a certain bit is set 
// OR a certain other bit is set.  An <code>AdvancedCriteria</code> like this would be 
// required (note that the choice of "equals" as the operator in these clauses is completely
// arbitrary - as discussed above, the operator is ignored when we have a 
// <code>customCriteriaExpression</code>, so it would make no difference to the end product 
// if we had used different operators here):<pre>
//   { _constructor: "AdvancedCriteria", operator: "or", criteria: [
//       { fieldName: "myField", operator: "equals", value: 32 },
//       { fieldName: "myField", operator: "equals", value: 64 },
//     ]
//   }       
// </pre>
// If you used the <code>customCriteriaExpression</code> from above unchanged with this criteria,
// the following SQL would be generated (in the MySQL case):
// <p>
// <code>SELECT myField, myOtherField FROM myTable WHERE myField & null = null OR myField & null = null</code>
// <p>
// This is because <code>$criteria</code> is a variable that only works correctly with simple
// criteria.  If you changed the <code>$criteria</code> references in the expression to 
// <code>$advancedCriteria</code>, the generated SQL would be:
// <p>
// <code>SELECT myField, myOtherField FROM myTable WHERE myField & 32 = 32 OR myField & 32 = 32</code>
// <p>
// This is because <code>$advancedCriteria</code> always returns the first value it finds for
// a particular field, as described above.  However, if you change the expression so it reads:
// <p>
// <code>&nbsp;&nbsp;&nbsp;myField & $criteriaValue = $criteriaValue</code>
// <p>
// you will now get the correct SQL generated:
// <p>
// <code>SELECT myField, myOtherField FROM myTable WHERE myField & 32 = 32 OR myField & 64 = 64</code>
// <p>
// Similar to <code>$criteriaValue</code>, we provide a special variable, 
// <code>$criteriaOperator</code>.  This is intended for use in Velocity expressions, so that 
// your <code>customCriteriaExpression</code> can process different operators in different ways.
// How you choose to interpret an operator is entirely up to you; this variable is both 
// advanced and rarely used.  This example is contrived and does not show a usage that would be 
// remotely likely in a real application, but it does show how to use the variable in a Velocity
// expression:
// <p>
// <code>&nbsp;&nbsp;&nbsp;myField #if ($criteriaOperator == "greaterThan") & #else | #end $criteriaValue = $criteriaValue</code>
// <p>
// For simple criteria, note that <code>$criteriaOperator</code> will vary depending on field
// type and the +link{type:TextMatchStyle,textMatchStyle} in force, as follows:<ul>
// <li>Text fields with textMatchStyle "substring" - "iContains"</li>
// <li>Text fields with textMatchStyle "startsWith" - "iStartsWith"</li>
// <li>All other cases - "equals"</li>
// </ul>
// You can use Velocity expressions in your <code>customCriteriaExpression</code>s, and the 
// +link{group:velocitySupport,usual context variables} are available.  Note that the results
// of Velocity evaluations are automatically quoted and escaped by default; this behavior can
// be switched off - see +link{autoQuoteCustomExpressions,autoQuoteCustomExpressions}, and 
// particularly note the warnings about its use.
// <p>
// Note that this property only applies to users of the SmartClient server using dataSources of
// type "sql".
//
// @see customSelectExpression
// @see customUpdateExpression
// @see customInsertExpression
// @see autoQuoteCustomExpressions
// @serverDS only
// @visibility customSQL
//<

//> @attr dataSourceField.autoQuoteCustomExpressions (boolean : true : IR)
// If explicitly set to false, switches off automatic quoting and escaping of values in
// custom expressions that are derived from Velocity evaluations.
// <p>
// By default, any part of a +link{customSelectExpression,customSelectExpression},
// +link{customCriteriaExpression,customCriteriaExpression},
// +link{customUpdateExpression,customUpdateExpression} or 
// +link{customInsertExpression,customInsertExpression} that comes ultimately from evaluating
// a Velocity expression, will be automatically quoted and escaped according to the syntax 
// of the underlying database.  We do this because "raw" values are vulnerable to 
// <a href="http://en.wikipedia.org/wiki/SQL_injection">SQL injection attacks</a>.
// <p>
// However, it is possible that the exact SQL logic you desire for a custom expression cannot
// be expressed without access to the raw, unquoted value.  To work around this, you can either
// use the special +link{group:velocitySupport,$rawValue} context variable, or you can set this
// property to false to switch off auto-quoting completely for all custom expressions on this
// field.
// <P>
// <b>Warning</b>:  Bear in mind that it is <b>dangerous</b> to use raw values.  There are some
// cases where using the raw value is necessary, but even so, all such cases are likely to be
// vulnerable to injection attacks.  Generally, the presence of <code>$rawValue</code> in a 
// custom expression, or any fields with <code>autoQuoteCustomExpressions: false</code> 
// specified, should be viewed as a red flag.
//
// @see customSelectExpression
// @see customUpdateExpression
// @see customInsertExpression
// @see customCriteriaExpression
// @serverDS only
// @visibility customSQL
//<


//> @attr dataSourceField.nativeName (string : null : IR)
// Only applicable to "sql" dataSources. If set, this property indicates that the field is 
// bound to the column named in the property.  It is used to bind a DataSourceField to a 
// database column with a different name.
// <p>
// <code>nativeName</code> is useful for disambiguating in cases where you have customized 
// SQL.  For example, assume you are joining two tables - call them <code>product</code> and
// <code>supplier</code> - that both contain a column called "name".  Only one field in the 
// DataSource can be called "name"; to disambiguate, we need to specify both 
// +link{dataSourceField.tableName,tableName} and <code>nativeName</code>.  A field declaration 
// like this would be needed:<br>
// <pre>    &lt;field name="supplierName" type="text" tableName="supplier" nativeName="name" /></pre>
// <p>
// You may also wish to use this property to map legacy column names to more meaningful field
// names in the DataSource.  For example:<br>
// <pre>    &lt;field name="productName" type="text" nativeName="PRDNM" /></pre>
//
// @see tableName
// @serverDS only
// @visibility customSQL
//<


//> @attr dataSourceField.tableName (String : null : IR)
// Only applicable to "sql" dataSources, setting <code>tableName</code> indicates that this
// field will be coming from another table by way of a 
// +link{group:customQuerying,customized SQL query}.
// <P>
// By default, the SQL generator will <b>not</b> include this field in generated SQL for "add"
// or "update" operations.  It will include the field in the SELECT clause and WHERE clause of
// customized "fetch" operations, where it is expected that you will make the table available
// via a custom &lt;tableClause&gt; as shown in +explorerExample{largeValueMapSQL,this sample}.
// <P>
// The table name will also be used to qualifying the column name unless
// +link{DataSource.qualifyColumnNames} or +link{OperationBinding.qualifyColumnNames} has been
// disabled.
// <p>
// <b>Note:</b> This property is only required when including fields from tables other than the
// default table specified by +link{DataSource.tableName}.
//
// @see nativeName
// @serverDS only
// @visibility customSQL
// @example largeValueMapSQL
//<

//> @method dataSourceField.exportForceText (ForceTextApproach : null : IR)
// When using +link{dataSource.recordsAsText}, what approach (if any) should be used to force
// values to be intepreted as text instead of heuristically parsed as dates, times or other
// structured types.
// @visibility external
//<

//> @attr dataSourceField.sqlStorageStrategy (String : null : IR)
// For certain field types, indicates the strategy to be used to store values to the underlying
// SQL column.
// <P>
// <b>Fields of type "boolean"</b><br>
// The default strategy for boolean fields is to assume the underlying type of the field is 
// text and store boolean true and false values as the character strings "true" and "false".  
// The following additional strategies are available:
// <ul>
// <li>"number" or "integer": store true as 1, false as 0
// <li>"singleChar10": store true as "1", false as "0"
// <li>"singleCharYN": store true as "Y", false as "N"
// <li>"singleCharTF": store true as "T", false as "F"
// <li>"singleChar": store as a single character, using the values defined by 
// +link{dataSourceField.sqlTrueValue,sqlTrueValue} and 
// +link{dataSourceField.sqlFalseValue,sqlFalseValue}
// <li>"string": store the entire values defined by 
// +link{dataSourceField.sqlTrueValue,sqlTrueValue} and 
// +link{dataSourceField.sqlFalseValue,sqlFalseValue}
// </ul>
// <P>
// <b>Fields of type "date", "time", and "datetime"</b><br>
// The default strategy for these types is to use a native type that supports date and 
// time values to the nearest second; the actual type used varies by database.  The following
// additional strategies are available:
// <ul>
// <li>"nativeDate": Use a native SQL DATE type</li>
// <li>"number" or "integer": Store the value as a plain numeric value</li>
// <li>"text": Store the value as a plain text value</li>
// </ul>
// The "number" and "text" strategies for date/time/datetime fields are provided to allow transparent
// two-way mapping of legacy date values; you would not ordinarily use them in the absence of a
// legacy database.  In both cases, the actual format of the number or text string is determined
// by specifying a +link{dataSourceField.sqlDateFormat,sqlDateFormat} on the field; if no 
// format is specified, the framework defaults to "yyyyMMdd".
// <P>
// <P>
// <b>Fields of type "text"</b><p>
// <em>NOTE: This section is only applicable for users of Microsoft SQL Server.  The "ntext"
// strategy is ignored for all other databases.</em><p>
// If you are integrating with Microsoft SQL Server, you may run into an issue with that database
// and double-byte character sets.  Although the documentation indicates that string values are 
// sent to the database as Unicode by default, they may undergo conversion at the database end, 
// depending on the collation settings of the database.  
// <a href=http://support.microsoft.com/kb/239530>This Microsoft support article</a> describes
// the situation, which is that Unicode is converted to the code page of the database before
// being used, in order to "provide backward compatibility with existing applications".
// <P>
// To work around this, we provide an <code>sqlStorageStrategy</code> of "ntext".  Fields that
// declare this strategy are inserted into queries using the "N syntax", like so:<br>
// <code>&nbsp;&nbsp;&nbsp;UPDATE MyTable SET MyTextColumn = N'some text' WHERE MyPK = N'some key'</code>
// <P>
// This syntax overrides defaults and forces the database to treat strings as Unicode.
// <P>
// Note that declaring a field to be of +link{type:FieldType,type} "ntext" is exactly 
// equivalent to specifying type "text" and an <code>sqlStorageStrategy</code> of "ntext".  
// If you declare a field of type "ntext" when the back-end database is something other than 
// SQL Server, it will be treated as a normal field of type "text".
// <p>
// This property only applies to users of the SmartClient server using dataSources of
// type "sql".
//
// @serverDS only
// @visibility customSQL
//< 

//> @attr dataSourceField.sqlDateFormat (String : null : IR)
// For fields of type "date", "time", and "datetime" that specify an
// +link{dataSourceField.sqlStorageStrategy,sqlStorageStrategy} of "number" or "text", this
// property allows you to specify the format SmartClient should use to map to/from the 
// underlying value stored in the database.  The string provided must be a valid 
// <a href=http://download.oracle.com/javase/1.4.2/docs/api/java/text/SimpleDateFormat.html>
// SimpleDateFormat pattern string</a>.
// <P>
// For fields of type "datetime" and "time" that specify an
// +link{dataSourceField.sqlStorageStrategy,sqlStorageStrategy} of "number", two special
// sqlDateFormat values are supported:
// <ul>
//   <li>"epoch" - The datetime or time value is stored as the number of full seconds that have
//       elapsed since January 1, 1970, 00:00:00 GMT.</li>
//   <li>"epochms" - The datetime or time value is stored as the number of milliseconds that
//       have elapsed since January 1, 1970, 00:00:00 GMT.</li>
// </ul>
// <P>
// This property and the "number" and "text" strategies for date/datetime fields are provided
// to allow transparent two-way mapping of legacy date values that are stored in numeric or
// text columns respectively rather than a true "date" column.  You would not ordinarily use them
// in the absence of a legacy database.
// <P>
// Note that this property only applies to users of SmartClient server and data sources of
// type "sql".
//
// @serverDS only
// @visibility customSQL
//< 

//> @attr dataSourceField.sqlTrueValue (String : null : IR)
// For fields of type "boolean" that specify a 
// +link{dataSourceField.sqlStorageStrategy,sqlStorageStrategy} of "singleChar", this is the
// value to persist for true values.  Note that the common use cases of storing booleans as 
// T/F, Y/N and 1/0 are already catered for without the need for this property.<br>
// For fields of type "boolean" that specify a 
// +link{dataSourceField.sqlStorageStrategy,sqlStorageStrategy} of "string", this is the value
// to persist for true values that require using the entire value.
//
// @see dataSourceField.sqlStorageStrategy
// @see dataSourceField.sqlFalseValue
// @serverDS only
// @visibility customSQL
//< 

//> @attr dataSourceField.sqlFalseValue (String : null : IR)
// For fields of type "boolean" that specify a 
// +link{dataSourceField.sqlStorageStrategy,sqlStorageStrategy} of "singleChar", this is the
// value to persist for false values.  Note that the common use cases of storing booleans as 
// T/F, Y/N and 1/0 are already catered for without the need for this property.<br>
// For fields of type "boolean" that specify a 
// +link{dataSourceField.sqlStorageStrategy,sqlStorageStrategy} of "string", this is the value
// to persist for false values that require using the entire value.  
//
// @see dataSourceField.sqlStorageStrategy
// @see dataSourceField.sqlTrueValue
// @serverDS only
// @visibility customSQL
//< 
    
    
//> @attr dataSourceField.nullReplacementValue (Any : null : IR)
// If +link{DataSource.noNullUpdates} is set, the value to use for this field when a null
// value is assigned to it on an update operation.  This property overrides the DataSource-level
// properties +link{DataSource.nullStringValue,nullStringValue}, 
// +link{DataSource.nullIntegerValue,nullIntegerValue}, 
// +link{DataSource.nullFloatValue,nullFloatValue} and 
// +link{DataSource.nullDateValue,nullDateValue}, for this field only.
// <p>
// NOTE: If you use this property, you are responsible for ensuring that the replacement value
// is appropriate and valid.
// @see DataSource.noNullUpdates
// @see DataSource.nullStringValue
// @see DataSource.nullIntegerValue
// @see DataSource.nullFloatValue
// @see DataSource.nullDateValue
// @serverDS allowed
// @visibility external
//<



//> @attr   dataSourceField.ignoreTextMatchStyle    (Boolean : null : IRW)
// <b>NOTE: </b>Only applicable to +link{clientOnly,clientOnly} DataSources and the built-in 
// +link{group:sqlDataSource,SQL DataSource} available in Pro, Power and Enterprise versions
// of SmartClient.
// <p>
// Use this flag to inhibit the normal use of +link{type:TextMatchStyle} for this field.  A 
// field with this flag set will always be tested for exact equality in generated queries, 
// even for filter-style queries where normal behavior would be to use a substring match or 
// similar.
//
// @serverDS only
// @visibility external
//<    

// ---------------------------------------------------------------------------------------
// client-side only sorting (canSortClientOnly)

//> @attr   dataSourceField.canSortClientOnly    (boolean : false : IRW)
// When true, this field can only be used for sorting if the data is entirely client-side.
// 
// @serverDS allowed
// @visibility external
//<

//> @attr dataSourceField.storeWithHash (HashAlgorithm : null : IR)
// If set, causes the field to be securely hashed before saving on an "add" or "update" 
// operation.
//
// @serverDS only
// @visibility external
//<

//> @type HashAlgorithm
// Algorithms to use when hashing field values (see +link{DataSourceField.storeWithHash})
//
// @value "MD5"      Message Digest algorithm 5 
// @value "SHA"      Secure Hashing Algorithm (SHA-1)
// @visibility external
//<

// ---------------------------------------------------------------------------------------
// File upload and other binary data properties

//> @attr   dataSourceField.maxFileSize    (Integer : null : IRW)
// For fields of a type that are represented by binary data initially uploaded from a file
// (currently "binary" and "imageFile", see +link{type:FieldType}), this sets the maximum 
// file size allowed, in bytes.
//
// @serverDS only
// @visibility external
//<

//> @attr dataSourceField.encodeInResponse (boolean : null : IRW)
// When set for a field with binary data, binary data will be delivered as a Base64 encoded
// string.  
// <P>
// When <code>encodeInResponse</code> is not set normal responses will not contain values for
// binary fields at all.  Instead, UI components, such as +link{class:TileGrid} issue a second
// binary fetch for binary data when appropriate, and APIs such as +link{downloadFile()} and
// +link{viewFile()} can be used to programmatically trigger downloads.
// <p>
// Only applicable to fields of a type that are represented by binary data (currently "binary"
// and "imageFile", see +link{type:FieldType}).
//
// @serverDS only
// @visibility external
//<


// ---------------------------------------------------------------------------------------
// Presentation Mode

//> @attr dataSourceField.fieldTreeId (String : null : IR)
// Value for the +link{tree.idField} used when +link{dataSource.showFieldsAsTree} is set.
//<

//>@attr dataSourceField.fieldTreeParentId (String : null : IR)
// Value for the +link{tree.parentIdField} used when +link{dataSource.showFieldsAsTree} is set.
//<

// Field Types
// --------------------------------------------------------------------------------------------

//> @type FieldType
// The types listed below are built-in types that +link{DataBoundComponent,databound
// components} understand and treat specially (using type-specific form controls, validators,
// formatters, sorting logic, etc).  
// <P>
// You can declare custom types via +link{SimpleType,SimpleType.create()}, with settings that
// will influence DataBound components.  You can also create your own subclasses of databound
// components to add further custom, reusable behaviors based on field.type.
// <P>
// <code>field.type</code> can also be the ID of another +link{DataSource}, which allows you to
// model nested structures such as XML documents (in fact, +link{XMLTools.loadXMLSchema()}
// models XML schema in this way).  Nested DataSource declarations affect how XML and JSON data
// is deserialized into JavaScript objects in the 
// +link{group:clientDataIntegration,client-side integration} pipeline, so that you can load
// complex XML documents and have them deserialized into a correctly typed JavaScript object
// model.
// <P>
// Note: to declare related but <i>separate</i> objects, as in an "Account" object that can be
// related to both a "Contact" object and "Order" objects, use
// +link{dataSourceField.foreignKey}, <b>not</b> a nested structure declaration.
//
// @value "text"      Generic text, e.g. <code>"John Doe"</code>. This is the default field type.
//                    Use <code>field.length</code> to set length.
// @value "boolean"   A boolean value, e.g. <code>true</code>
// @value "integer"   A whole number, e.g. <code>123</code>
// @value "float"     A floating point (decimal) number, e.g. <code>1.23</code>
// @value "date"      A logical date, with no time value (such as a holiday or birthday).
//                    Represented on the client as a JavaScript <code>Date</code> object where
//                    time values are ignored.  See +link{group:dateFormatAndStorage} for more
//                    information on date display and serialization formats. See also
//                    +link{class:Date} for SmartClient extensions to
//                    the <code>Date</code> object.
// @value "time"      A time of day, with no date.  Represented on the client as a JavaScript
//                    <code>Date</code> object in UTC/GMT by default (see also
//                    +link{group:dateFormatAndStorage} and the +link{Time} class).
// @value "datetime"  A date and time, accurate to the second. Represented on the client as a 
//                    JavaScript <code>Date</code> object. See also
//                    +link{group:dateFormatAndStorage}
//                    and +link{class:Date} for
//                    SmartClient extensions to the <code>Date</code> object.
// @value "enum"      A text value constrained to a set of legal values specified by the
//                    field's +link{DataSourceField.valueMap,valueMap}, as though a
//                    +link{ValidatorType} of "isOneOf" had been declared.
// @value "intEnum"   An enum whose values are numeric.
// @value "sequence"  If you are using the SmartClient SQL datasource connector, a
//                    <code>sequence</code> is a unique, increasing whole number, incremented
//                    whenever a new record is added. Otherwise, <code>sequence</code> behaves
//                    identically to <code>integer</code>. This type is typically used with
//                    <code>field.primaryKey</code> to auto-generate unique primary keys.
//                    See also +link{dataSourceField.sequenceName}.
// @value "link"      A string representing a well-formed URL.  Some components
//                    will render this as an HTML link (using an anchor tag for example).    
// @value "image"     A string representing a well-formed URL that points to an
//                    image.  Some components will render an IMG tag with the value of this field
//                    as the 'src' attribute to render the image.
// @value "binary"    Arbitrary binary data.  When this field type is present, three additional
// fields are automatically generated.  They are: &lt;fieldName&gt;_filename, &lt;fieldName&gt;_filesize, and
// &lt;fieldName&gt;_date_created where &lt;fieldName&gt; is the value of the <code>name</code>
// attribute of this field.  These fields are marked as
// +link{attr:dataSourceField.hidden}<code>:true</code> to suppress their rendering by default.  You
// can show one or more of these fields by specifying the field with a <code>hidden:false</code>
// override in the fields array of the databound component.
// <i>Stream / view file support for custom DataSources</i>: a custom DataSource or +link{DMI} 
// must implement the "viewFile" and "downloadFile" operationTypes and return a single Record
// with a byte[] as the field value for the binary field.  For more detail see
// the overview of +link{group:binaryFields,Binary Fields}.
//
// @value "imageFile" Binary data comprising an image.  Causes +link{ViewFileItem} to be used
//                    when the field is displayed in a form, allowing the image to optionally
//                    be displayed inline.
// @value "any"       Fields of this type can contain any data value and have no default 
//                    formatting or validation behavior. This is useful as the 
//                    +link{SimpleType.inheritsFrom,parent type} for SimpleTypes
//                    where you do not want any of the standard validation or formatting logic
//                    to be inherited from the standard built-in types.
// @value "custom"    Synonymous with "any".
// @value "modifier"  Fields of this type are automatically populated by the SmartClient Server 
//                    with the current authenticated userId as part of "add" and "update" 
//                    operations.  By default, fields of this type are hidden and not editable;
//                    the server ignores any value that the client sends in a field of this type.
//                    Note that the "authenticated user" can be set explicitly on the server-side 
//                    <code>RPCManager</code> using the setUserId() method, or it can come from
//                    the servlet API if you are using its built-in authentication scheme.  See
//                    the server-side Javadocs for <code>RPCManager</code>.
// @value "modifierTimestamp"  Fields of this type are automatically populated by the SmartClient
//                    Server with the current date and time as part of "add" and "update" 
//                    operations.  By default, fields of this type are hidden and not editable;
//                    the server ignores any value that the client sends in a field of this type.
// @value "creator"   Fields of this type are automatically populated by the SmartClient Server 
//                    with the current authenticated userId as part of "add" operations.
//                    By default, fields of this type are hidden and not editable;
//                    the server ignores any value that the client sends in a field of this type.
//                    The notes about type "modifier" also apply to fields of this type.
// @value "creatorTimestamp"  Fields of this type are automatically populated by the SmartClient
//                    Server with the current date and time as part of an "add" operation (when
//                    the record is first created).  By default, fields of this type are hidden
//                    and not editable; the server ignores any value that the client sends in a
//                    field of this type.
// @value "password"  Same as "text", but causes +link{PasswordItem} to be used by default for
//                    editing (hides typed-in value).
// @value "ntext"     A special field type specifically for use with Unicode data in conjunction
//                    with the Microsoft SQL Server database.  Field type "ntext" implies the 
//                    use of +link{DataSourceField.sqlStorageStrategy,sqlStorageStrategy} "ntext";
//                    other than that, this type is identical to "text"
// @value "localeInt" An integer number with locale-based formatting, e.g. <code>12,345,678</code>.
//                    See +link{group:localizedNumberFormatting,Localized Number Formatting}
//                    for more info.
// @value "localeFloat" A float number with locale-based formatting, e.g. <code>12,345.67</code>.
//                    See +link{group:localizedNumberFormatting,Localized Number Formatting}
//                    for more info.
// @value "localeCurrency" A float number with locale-based formatting and using currency
//                    symbol, e.g. <code>$12,345.67</code>.
//                    See +link{group:localizedNumberFormatting,Localized Number Formatting}
//                    for more info.
//
//
// @see type:ListGridFieldType
// @see type:FormItemType
// @serverDS allowed
// @visibility external
//<


// Titles
// ---------------------------------------------------------------------------------------

//> @attr DataSource.title (string : dataSource.ID : IRW)
// User-visible name for this DataSource.
// <P>
// For example, for the supplyItem DataSource, "Supply Item".
// <P>
// Defaults to <code>dataSource.ID</code>.
// @group titles
// @serverDS allowed
// @visibility external
//<

//> @attr DataSource.pluralTitle (string : dataSource.ID : IR)
// User-visible plural name for this DataSource.
// <P>
// For example, for the supplyItem DataSource, "Supply Items".
// <P>
// Defaults to (<code>dataSource.title</code> or <code>dataSource.ID</code>) + "s".
// @group titles
// @serverDS allowed
// @visibility external
//<

//> @groupDef dsSpecialFields
// A DataSource allows you to declare that certain fields are the most important fields to 
// show to a user in situations where space is limited, and only one or a few fields can be
// reasonably shown.
// <P>
// In the table below these special fields are summarized, along with their meaning, and
// examples of which field would be most appropriate from several example DataSources.
// <P>
// <table cellPadding=2 class="normal" border=1>
// <tr style="font-weight:bold;">
//  <td rowSpan="2">Attribute</td>
//  <td colSpan="3">Example DataSource field</td>
//  <td rowSpan="2">Meaning</td>
// </tr><tr style="font-style:italic;">
//  <td>employee</td><td>emailMessage</td><td>stockItem</td>
// </tr><tr>
//  <td>titleField</td><td>name</td><td>subject</td><td>itemName</td>
//      <td>primary label for the record as a while</td>
// </tr><tr>
//  <td>infoField</td><td>job</td><td>sender</td><td>category</td>
//      <td>second most pertinent piece of textual information</td>
// </tr><tr>
//  <td>dataField</td><td>salary</td><td>date</td><td>price</td>
//      <td>most pertinent numeric, date or enum (eg status) field</td>
// </tr><tr>
//  <td>descriptionField</td><td>bio</td><td>messageBody</td><td>description</td>
//      <td>descriptive long text field</td>
// </tr><tr>
//  <td>iconField</td><td>photo</td><td>statusIcon</td><td>thumbnail</td>
//      <td> an image or icon to accompany the title field</td>
// </tr></table>
// <P>
// Examples of the use of these fields include the +link{TileGrid}'s default choice of fields,
// and the +link{EventHandler.setDragTracker(),drag tracker} that follows the mouse cursor when
// data is being dragged between grids.
//
// @visibility external
//<


//> @attr dataSource.titleField (string : see below : IR)
// Best field to use for a user-visible title for an individual record from this dataSource.
// <p>
// For example, for a DataSource of employees, a "full name" field would probably most clearly
// label an employee record.
// <p>
// If not explicitly set, titleField looks for fields named "title", "label", "name", and "id"
// in that order.  If a field exists with one of those names, it becomes the titleField.  If not,
// then the first field is designated as the titleField.
//
// @group titles, dsSpecialFields
// @serverDS allowed
// @visibility external
//<

//> @attr dataSource.iconField (string : see below : IR)
// Designates a field of +link{FieldType,type}:"image" as the field to use when rendering a
// record as an image, for example, in a +link{TileGrid}.
// <p>
// For example, for a DataSource of employees, a "photo" field of type "image" should be
// designated as the iconField.
// <p>
// If not explicitly set, iconField looks for fields named "picture", "thumbnail", "icon",
// "image" and "img", in that order, and will use any of these fields as the iconField if it
// exists and has type "image". 
// <P>
// To avoid any field being used as the iconField, set iconField to <code>null</code>.
// 
// @group dsSpecialFields
// @serverDS allowed
// @visibility external
//<

//> @attr dataSource.infoField (String : null : IR)
// Name of the field that has the second most pertinent piece of textual information in the
// record, for use when a +link{DataBoundComponent} needs to show a short summary of a record.
// <p>
// For example, for a DataSource of employees, a "job title" field would probably be the second
// most pertinent text field aside from the employee's "full name".
// <p>
// Unlike +link{titleField}, infoField is not automatically determined in the absence of an
// explicit setting.
// @group dsSpecialFields
// @serverDS allowed
// @visibility external
//<


//> @attr dataSource.dataField (String : null : IR)
// Name of the field that has the most pertinent numeric, date, or enum value, for use when a
// +link{DataBoundComponent} needs to show a short summary of a record.
// <P>
// For example, for a DataSource of employees, good choices might be the "salary" field, "hire
// date" or "status" (active, vacation, on leave, etc).
// <p>
// Unlike +link{titleField}, dataField is not automatically determined in the absence of an
// explicit setting.
// @group dsSpecialFields
// @serverDS allowed
// @visibility external
//<

//> @attr dataSource.descriptionField (String : null : IR)
// Name of the field that has a long description of the record, or has the primary text data
// value for a record that represents an email message, SMS, log or similar.
// <p>
// For example, for a DataSource representing employees, a field containing the employee's
// "bio" might be a good choice, or for an email message, the message body.
// <p>
// If descriptionField is unset, it defaults to any field named "description" or "desc" in the
// record, or the first long text field (greater than 255 characters) in the record, or null if
// no such field exists.
// @group dsSpecialFields
// @serverDS allowed
// @visibility external
//<

//> @attr dataSource.childrenField (string : null : [IR])
// fieldName for a field in the dataSource expected to contain an explicit array of child nodes.
// Enables loading a databound tree as a hierarchical data structure, rather than a flat list of
// nodes linked by foreignKey.<br>
// Note this is an alternative to setting +link{dataSourceField.childrenProperty} directly on
// the childrenField object.<P>
// By default the children field will be assumed to be +link{dataSourceField.multiple, multiple},
// for XML databinding. This implies that child data should be delivered in the format:
// <pre>
//      &lt;childrenFieldName&gt;
//          &lt;item name="firstChild" ...&gt;
//          &lt;item name="secondChild" ...&gt;
//      &lt;/childrenFieldName&gt;
// </pre>
// However data may also be delivered as a direct list of <code>childrenFieldName</code> 
// elements:
// <pre>
//      &lt;childrenFieldName name="firstChild" ...&gt;
//      &lt;childrenFieldName name="secondChild" ...&gt;
// </pre>
// If you want to return your data in this format, you will need to explicitly set 
// <code>multiple</code> to false in the appropriate dataSource field definition.
//
// @see dataSourceField.childrenProperty
// @group dataSourceRelations
// @serverDS allowed
// @visibility external
//<

//> @groupDef serverDataSourceImplementation
// NOTE: This section includes references to server-side classes and methods; you can find the
// documentation for these in the server-side Javadocs.
// <p>
// Bear in mind the following points when working with DataSources (whether custom or 
// built-in) on the SmartClient Server:<ul>
// <li>There is no requirement or expectation that DataSource subclasses be threadsafe.  As 
// long as you use documented techniques for creating DataSource instances, the server framework
// will ensure that no single instance is used concurrently by more than one thread.</li>
// <li>DataSource instances can safely be retained for use multiple times in a given HTTP 
// request; this presents no thread-safety issues unless you specifically introduce them by,
// eg, creating a new DSRequest and executing it in its own thread.  However, note that mixing
// +link{DataSource.autoJoinTransactions,transactional} and non-transactional operations on the 
// same DataSource instance in a single HTTP request can lead to connection leaks if you are 
// using the built-in Hibernate or JPA DataSource.</li>
// <li>If the +link{server_properties,server.properties} attribute <code>datasources.pool.enabled</code>
// is set, DataSource instances are automatically cached in an in-memory pool.  This caching
// is smart: if the underlying <code>.ds.xml</code> file changes, cached instances are 
// discarded and rebuilt.</li>
// <li>If pooling is not switched on, DataSource instances are constructed new as required.
// Whether or not pooling is enabled, new DataSource instances are always constructed using
// cached configuration for better performance.  The configuration file (<code>.ds.xml</code>
// file) is read the first time the DataSource is requested; thereafter, we will only re-read
// the configuration file if its last-changed timestamp changes.  This scheme combines the 
// performance benefits of caching with the with the convenience of being able to change a 
// <code>.ds.xml</code> file and pick up the changes without restarting the server.</li>
// <li>The safe way to obtain a DataSource instance is to use 
// <code>RPCManager.getDataSource()</code>.  If pooling is enabled, this is the only way to 
// obtain a DataSource instance with the guarantee that it will be returned to the pool at 
// the end of request processing.</li>
// <li>If you are using the <code>DynamicDSGenerator</code> to provide DataSources to the 
// framework dynamically:<ul>
// <li>You are expected to use one of the <code>DataSource.fromXML()</code> APIs to construct
// your DataSources from an XML document or String of XML text</li>
// <li>Do not use normal Java instantiation techniques like <code>new DataSource()</code>, as 
// this will introduce thread-safety issues</li>
// <li>Return a new DataSource instance each time - returning the same DataSource twice will
// introduce thread-safety issues.  If you are concerned about performance, ensure pooling is
// enabled as described above - this will mean that the DataSource you return will be
// automatically cached and re-used, so your DynamicDSGenerator will only be called enough
// times to populate the pool.  If you are still concerned about performance, cache the XML
// String that you pass to <code>DataSource.fromXML()</code>.</li>
// <li>For many applications of <code>DynamicDSGenerator</code>, pooling is inappropriate
// because the returned DataSource for a given name might be different each time the generator
// is called.  For this reason, pooling of dynamic DataSources is disabled by default.  To
// enable pooling for dynamically-generated DataSources, set the +link{server_properties,server.properties} 
// flag <code>poolDynamicDataSources</code> to true.  NOTE: Here, "dynamic DataSource" means 
// a DataSource whose name would cause the framework to invoke a DynamicDSGenerator, which 
// doesn't necessarily mean that the generator would actually create a dynamic DataSource.  As
// the server-side documentation for <code>DynamicDSGenerator.getDataSource()</code> states,
// a generator can simply return null to decline the opportunity to create a dynamic DataSource.
// Therefore, if the proper operation of pooling is important to you, avoid patterns of 
// <code>DynamicDSGenerator</code> usage that involve registering more generically than you 
// need to.  In particular, avoid the <code>addDynamicDSGenerator()</code> signature that 
// does not take a prefix or regex parameter - this will cause the pooling subsystem to regard
// <b>all</b> DataSources as dynamic, and will effectively disable all DataSource pooling.
// </li></ul>
// </li></ul>
//
// @treeLocation Client Reference/Data Binding/DataSource
// @title Notes on Server-side DataSource Implementations
// @visibility external
//<

//> @attr dataSource.serverConstructor (String : null : IR)
//
// This property allows you to write and use custom DataSource subclasses on the server, by 
// specifying either <ul>
// <li>the fully-qualified name of the DataSource subclass that should be instantiated
// server-side for this dataSource, or</li>
// <li>the token "spring:" followed by a valid Spring bean ID, if you wish to instantiate 
// your custom dataSource object using Spring dependency injection.  For example, 
// <code>"spring:MyDataSourceBean"</code>.  See also +link{group:serverInit} for special
// concerns with framework initialization when using Spring.</li>
// </ul>
// <p>
// One reason you might wish to do this would be to override the validate() method to provide 
// some arbitrary custom validation (such as complex database lookups, validation embedded in 
// legacy applications, etc).  It is also possible - though obviously a more substantial task -
// to override the execute() method in your custom DataSource.  This is one way of creating 
// a completely customized DataSource implementation.
// <p>
// <b>Note:</b> If you use this property, you are responsible for making sure that it refers to
// a valid server-side class that extends <code>com.isomorphic.datasource.BasicDataSource</code>,
// or to a Spring bean of the same description.
// If your implementation relies on methods or state only present in certain specialized 
// subclasses of DataSource (for example, you want the normal behavior and features of a 
// HibernateDataSource, but with a specialized validate() method), then you should extend the 
// subclass rather than the base class.
// <P>
// NOTE: Please take note of the points made in 
// +link{group:serverDataSourceImplementation,this discussion} of caching and thread-safety
// issues in server-side DataSources.
//
// @serverDS only
// @visibility external
//<

//> @attr dataSource.qualifyColumnNames (Boolean : true : IR)
//
// For dataSources of +link{serverType,serverType} "sql", determines whether we qualify column
// names with table names in any SQL we generate.  This property can be overridden on specific
// operationBindings.
//
// @see OperationBinding.qualifyColumnNames
// @serverDS allowed
// @visibility external
//<

//> @attr dataSource.validateRelatedRecords (boolean : null : IR)
//
// If true, indicates that the SmartClient Server should automatically apply a 
// +link{ValidatorType} of "hasRelatedRecord" to every field 
// on this dataSource that has a +link{DataSourceField.foreignKey,foreignKey} defined.
//
// @visibility external
//<

//> @attr dataSource.beanClassName (String : null : IR)
//
// This property has different meanings depending on the +link{dataSource.serverType,serverType}:
// <p>
// <b>For SQL DataSources (DataSources with serverType "sql")</b><br/>
// If set, results from the database will be used to create one instance of the indicated Java
// bean per database row.  Otherwise a Map is used to represent each row retrieved from SQL.
// <P>
// With this feature active, a DSResponse from this DataSource will contain a Collection of
// instances of the indicated <code>beanClassName</code>, available via DSResponse.getData().
// This creates a couple of possibilities:
// <P>
// <dl>
// <dt>Add business logic for derived properties, such as computed formulas</dt>
// <dd>For example, declare a DataSourceField named "revenueProjection".  By default this
// field will call getRevenueProjection() on your bean to retrieve the value to send to the
// client.  Your implementation of getRevenueProjection() could apply some kind of formula to
// other values loaded from the database.</dd>
// <dt>Call business logic on retrieved beans via DMI</dt>
// <dd>By adding a +link{dmiOverview,DMI} method that calls DSRequest.execute() to retrieve a DSResponse,
// you have an opportunity to call business logic methods on the beans representing each
// row affected by the DSRequest.  For example, notify a related BPEL process of changes to
// certain fields.</dd>
// </dl>
// <P>
// By using <code>beanClassName</code> on a specific +link{operationBinding}, you can:
// <ul>
// <li>Use a bean to represent your data only when it matters; for example, avoid the overhead
//     of using a bean for "fetch" operations, but do use a bean for "update" operations so
//     that you can execute relevant business logic after the update completes.</li>
// <li>Skip the use of beans for complex reporting queries that produce results unrelated to
//     your persistent object model.  Set beanClassName to blank ("") on a specific
//     operationBinding to override DataSource.beanClassName for that specific operation.</li>
// <li>For SQL joins that produce additional data fields, use a special, operation-specific
//     bean that represents a join of multiple entities and contains business logic specific
//     to that joined dataset</li>
// </ul>
// <P>
// Note that setting <code>beanClassName</code> has no effect on the DSRequest.  However,
// +link{dmiOverview,DMI} has a built-in facility for populating a bean with the inbound
// +link{DSRequest.data} - just declare the bean as a method argument.
// <p>
// <b>For generic DataSources (DataSources with serverType "generic")</b><br>
// +link{group:visualBuilder,Visual Builder} sets this property when it creates a generic 
// DataSource using the Javabean Wizard.  It has no built-in server-side effects.
// <p/>
// <b>For Hibernate DataSources (DataSources with serverType "hibernate")</b><br>
// The name of the Java bean or POJO class that is mapped in Hibernate.  This will typically 
// be the fully-qualified class name - eg <code>com.foo.MyClass</code> - but it may be the 
// simple class name - just <code>MyClass</code> - or it may be some other value.  It all 
// depends on how your classes have been mapped in Hibernate.
// <p>
// Note:  If you are intending to use Hibernate as a data-access layer only, 
// you do not need to create Hibernate mappings or Java objects: SmartClient will generate
// everything it needs on the fly.
// <p>
// <b>For JPA DataSources (DataSources with serverType "jpa" or "jpa1")</b><br/>
// The fully qualified class name of the JPA annotated entity.
// <p>
// <b>NOTE for Hibernate and JPA users:</b> When you use JPA, or use Hibernate as a full ORM
// system (ie, not just allowing SmartClient Server to drive Hibernate as a data access layer),
// the beans returned on the server-side are <b>live</b>.  This means that if you make any 
// changes to them, the ORM system will persist those changes.  This is true even if the beans
// were created as part of a fetch operation.
// <p>
// This causes a problem in the common case where you want to use a DMI or custom DataSource 
// implementation to apply some post-processing to the beans fetched from the persistent 
// store.  If you change the values in the beans directly, those changes will be persisted.
// <p>
// If you want to alter the data returned from a JPA or Hibernate persistent store as part of
// a fetch request just so you can alter what gets sent to the client, you can use the 
// server-side <code>DSResponse</code>'s <code>getRecords()</code> method.  This will return
// your bean data in "record" format - ie, as a List of Maps.  You can alter these records 
// without affecting your persistent store, and then call <code>setData()</code> on the 
// <code>DSResponse</code), passing the altered list of records.  See the server-side Javadocs
// for <code>DSResponse</code> for details of these two methods.
//
// @see OperationBinding.beanClassName
// @serverDS only
// @visibility external
//<

isc.DataSource.addMethods({
	init : function () {
        // serverType:"sql" implies iscServer format
        if (this.serverType == "sql") this.dataFormat = "iscServer";

        // make it unnecessary to set dataFormat="xml" when properties related to XML have
        // already been set
        if (this.dataFormat == "iscServer" &&
            (this.serviceNamespace != null || this.recordXPath != null)) this.dataFormat = "xml";
        
        // we don't currently support request queuing (multi-op) for anything other than the
        // iscServer format
        this.canQueueRequests = (this.dataFormat == "iscServer" || this.clientOnly);

        // if testData is set and cacheData isn't, set cacheData now, since testData is marked
        // as deprecated
        if (this.testData && !this.cacheData) this.cacheData = this.testData;
        // if cacheData is set and testData isn't, and we're in clientOnly mode, set testData
        // now, since internal code requires it but it's marked externally as deprecated
        else if (this.clientOnly) {
                this.cacheData = this.testData = this.cacheData || this.testData;
        }

        
        if (this.ID == null && this.id != null) this.ID = this.id;

		
        if (this.name == null) this.name = this.ID;

        // prevent the server definition of eg ValueMap from destroying certain built-in schema
        var existingDS = isc.DS.get(this.ID);
        if (existingDS && existingDS.builtinSchema) return existingDS;

        // make the DataSource a global variable if set to do so, and it doesn't belong to a
        // particular namespace
        // NOTE: when becoming global, only clobber other DataSources, never a Class or any
        // other kind of global.  Otherwise widget DataSources like Canvas clobber the
        // corresponding widget class.  Note we have to specially check for isA.ClassObject()
        // because the DataSource classObject and subclasses will return true when
        // isA.DataSource() is applied to them.
        var existingValue = window[this.ID];
        if (this.addGlobalId && this.addGlobalId != isc._false &&
            (!existingValue || 
                (!isc.isA.ClassObject(existingValue) && isc.isA.DataSource(existingValue)))) 
        {
            isc.ClassFactory.addGlobalID(this);
        }

        
        // if field is provided as an Array, make them into an index by name (so client-only
        // DataSources can use the array of fields format common to all components).  NOTE:
        // this is essentially Array.makeIndex, except we log collisions.
        var fields = this.fields;
        if (isc.isAn.Array(fields)) {
            if (isc.Canvas.validateFieldNames) {
                // loop through each field and check that each field name is a valid JavaScript identifier.
                isc.Canvas._validateFieldNames(fields, this);
            }

            var fieldIndex = {};
            for (var i = 0; i < fields.length; i++) {
                var field = fields[i];
                // if includeFrom is set, but name isn't pick up name from includeFrom property
                if (field.includeFrom != null) {
                    var split = field.includeFrom.split(".");
                    if (split == null || split.length  != 2) {
                        this.logWarn("Field has includeFrom specified as :"  + field.includeFrom
                            + " format not understood - clearing this property");
                        field.includeFrom = null;
                    } else {
                        if (field.name == null) field.name = split[1];
                    }
                }
                
                // last field wins on name collision.  
                if (fieldIndex[field.name] != null) {
                    this.logWarn("field.name collision: first field: " +
                                 this.echo(fieldIndex[field.name]) + 
                                 ", discarded field: " + this.echo(field));
                    continue;
                }
                field = this._prettyField(field);
                if (field.type) {
                    var type = this.getType(field.type);
                    if (type && type.fieldProperties) {
                        field = fields[i] = isc.addProperties({}, type.fieldProperties, field);
                    }
                }
                fieldIndex[field.name] = field;
            }
            this.fields = fieldIndex;
        }
	
		if (this.dataSourceType == isc.DataSource.VIEW) this.initViewSources();

        // register globally
        isc.DataSource.registerDataSource(this);
	},

    _prettyField : function (field) {
        if (isc.isA.String(field.validOperators)) {
            field.validOperators = [field.validOperators];
        }

        var undef;
        if ((field.foreignKey)
            && (field.nillable == undef)) {
            field.nillable = true;
        }

        return field;
    },
	
    // destroy global ID, if one was assigned.
    destroy : function () { 
        var ID = this.ID,
            DS = isc.DS;
        
        // remove from global scope
        if (ID && this == window[ID]) window[ID] = null;

        // remove from global registries
        if (DS._dataSources[ID] == this) DS._dataSources[ID] = null;
        if (DS._elements[ID] == this) DS._elements[ID] = null;
        if (DS._types[ID] == this) DS._types[ID] = null;
        this.Super("destroy", arguments);
    },
    

    // Result Sets
	// --------------------------------------------------------------------------------------------
    getResultSet : function (params) {
        var resultSetClass = isc.ClassFactory.getClass(this.resultSetClass || isc.ResultSet);
        
        if (!isc.isA.Class(resultSetClass)) {
            this.logWarn("getResultSet(): Unrecognized 'resultSetClass' property:" 
                         + resultSetClass + ", returning a standard isc.ResultSet.");
            resultSetClass = isc.ResultSet;
        }
        
        // flag resultSet as autoCreated, so we know to destroy() it when the component no
        // longer needs it
        return resultSetClass.create(params, {_autoCreated: true});
    },
    
    // Observable dataChanged methods.
    // ResultSets associated with this dataSource will observe these methods and use them
    // to handle cache-sync
    // Dont obfuscate: allow observers to pick up parameter names            
    dataChanged : function (dsResponse, dsRequest) {
//        alert("dataChanged: " + this);
        //!DONTOBFUSCATE
    },
    
    
    
    //> @method DataSource.updateCaches()
    // Causes any components using this DataSource to be notified of changes that have been made to
    // the remote dataset accessed via this DataSource, as though the provided DSResponse had just
    // successfully completed.  This will cause cache managers such as +link{ResultSet} or 
    // +link{ResultTree} to automatically update their caches, and components using such cache
    // managers to visually update to show modified data.
    // <P>
    // This API should be used when you have found out about changes made by other users or by
    // automatic processes.  For example, using the SmartClient +link{Messaging} system to receive
    // real-time updates via HTTP streaming, you may get updates that should affect a ListGrid
    // which is using a ResultSet to view a portion of a large dataset.  
    // <P>
    // The provided <code>DSResponse</code> should have
    // +link{dsResponse.operationType,operationType} "update", "add" or "remove" - there is no
    // way for a "fetch" response to meaningfully update arbitrary caches.  However, if you
    // have a list of updated records (possibly retrieved via +link{fetchData()}) you can call
    // <code>updateCaches()</code> multiple times with DSResponses of type "update" formed from
    // the list of records retrieved via <code>fetchData()</code>
    // <P>
    // To cause all components that have cache managers to drop their caches, provide a
    // DSResponse with +link{dsResponse.invalidateCache} set.
    // <P>
    // As an alternative to calling <code>updateCaches()</code> directly, if updates to other
    // DataSources occur as a result of server-side logic, you can use the server-side API
    // DSResponse.addRelatedUpdate(DSResponse) (Pro Edition and above), which ultimately calls
    // <code>updateCaches()</code> for you - see that method's documentation for details.
    // <P>
    // <b>NOTE:</b>: this API should <b>not</b> be used with a +link{clientOnly} DataSource,
    // because in this case, the "remote dataset" is actually within the browser.  Instead, 
    // +link{DataSource.updateData()}, addData() or removeData() can be called in order to both
    // change the dataset stored inside the browser and notify all cache managers.
    // <P>
    // If a DataSource has +link{cacheAllData} set and a full cache has been obtained, calling
    // <code>updateCaches</code> will automatically update the cache.
    // <P>
    // Note that the DSResponse provided to this method will <b>not</b> go through
    // +link{transformResponse()} or other processing that would normally occur for a
    // DSResponse resulting from a DSRequest sent by the application in this page.
    //
    // @param dsResponse (DSResponse) the provided DSResponse must minimally have 
    //                                dataSource, operationType, and data set
    // @param [dsRequest] (DSRequest) optional dsRequest.  If not specified, a DSRequest will be
    //                                automatically created based on the DataSource and operationType of
    //                                the DSResponse
    // @visibility external
    //<
	updateCaches : function (dsResponse, dsRequest) {
	    // If dsRequest is null, generate a dummy request with the appropriate operationType
	    // and dataSource
	    if (dsRequest == null) {
	        dsRequest = {
	            operationType:dsResponse.operationType,
	            dataSource:this
	        };
	        if (dsResponse.clientContext != null) {
	            dsRequest.clientContext = dsResponse.clientContext;
	        }
            if (dsResponse.internalClientContext != null) {
                dsRequest.internalClientContext = dsResponse.internalClientContext;
            }
	    } else {
            // ensure the dsRequest is marked with a DataSource.  Downstream code depends on this
            dsRequest.dataSource = dsRequest.dataSource || dsResponse.dataSource || this;
        }

		var updateData = dsResponse.data,
            forceCacheInvalidation = dsResponse.invalidateCache,
            responseCode = dsResponse.httpResponseCode;

        // check the response code for a valid empty file, in which case we just keep 
        // the client-side updates.
		if (!updateData && !forceCacheInvalidation 
            && !(responseCode >= 200 && responseCode < 300)) {
			//>DEBUG
			this.logWarn("Empty results returned on '" + dsRequest.operationType + 
                         "' on dataSource '" + dsRequest.dataSource + 
                         "', unable to update resultSet(s) on DataSource " + this.ID +
                         ".  Return affected records to ensure cache consistency.");
			//<DEBUG
			return;
		}
        
        // if we're caching all data and the cache is full, force refresh if data is invalid
        
        if (this.cacheAllData && this.hasAllData() && !isc.isAn.Array(updateData)) {
            this.invalidateCache();
        }

        this.dataChanged(dsResponse, dsRequest);
	},

    // ---------------------------------------------------------------------------------------
    // XML
    // ---------------------------------------------------------------------------------------

    //> @method dataSource.getLegalChildTags()
    // For a DataSource that describes a DOM structure, the list of legal child elements that can
    // be contained by the element described by this DataSource.
    // <p>
    // For a DataSource described by XML schema, this is the list of legal subelements <b>of
    // complexType</b> (elements of simpleType become DataSourceFields with atomic type).
    // <p>
    // Note that currently, if an XML schema file contains ordering constraints, DataSources
    // derived from XML Schema do not capture these constraints.
    //
    // @group xmlSchema
    // @visibility xmlBinding
    //<
    getLegalChildTags : function () {
        var fieldNames = this.getFieldNames(),
            complexFields = [];
        for (var i = 0; i < fieldNames.length; i++) {
            if (this.fieldIsComplexType(fieldNames[i])) complexFields.add(fieldNames[i]);
        }
        return complexFields;
    },

    // get the operationBinding for this operation.  The operationBinding is the bundle of 
    // properties specifying how to invoke a particular DataSource operation, eg "fetch".  
    // <BR>
    // In the absence of an operationBinding for a particular operation, the DataSource itself 
    // is used as the operationBinding, with the same set of properties supported - in this 
    // case you basically have a read-only DataSource, only capable of fetching.
    // <BR>
    // Takes either a dsRequest or an operationType and optional operationId.  
    getOperationBinding : function (operationType, operationId) { 
        if (operationType == null || this.operationBindings == null) return this;

        if (isc.isAn.Object(operationType)) {
            // passed a DSRequest 
            var dsRequest = operationType;
            operationType = dsRequest.operationType;
            operationId = dsRequest.operationId;
        }

        var schemaName;
        // look for a binding specific to the operationId (eg myFetchSchema) if passed
        if (operationId) {
            var operationBinding = this.operationBindings.find("operationId", operationId);
            if (operationBinding) return operationBinding;
        }
        // look for a binding for this operationType
        if (operationType) { 
            var operationBinding = this.operationBindings.find("operationType", operationType);
            if (operationBinding) return operationBinding;
        }
        return this;
    },

    getDataFormat : function (operationType, operationId) {
        return this.getOperationBinding(operationType, operationId).dataFormat || this.dataFormat;
    },
    
    // If preventHTTPCaching is set, use the RPCRequest.bypassCache attribute
    shouldBypassCache : function (operationType, operationId) {
        var preventCache = this.getOperationBinding(operationType, operationId).preventHTTPCaching;
        if (preventCache == null) preventCache = this.preventHTTPCaching;        
        return preventCache;
    },

    //> @method dataSource.copyRecord()
    // Copies all DataSource field values of a Record (including a TreeNode) to a new 
    // Record, omitting component-specific metadata such as selected state from grids, 
    // or parent folders for TreeNodes.
    // @param record (Record) The record to be copied. 
    // @return  (Record) A new copy of the record provided as an argument, with 
    // component-specific metata data removed.
    // @visibility external
    //<
    copyRecord : function (original) {
    	if (original == null) return null;

        var copy = {},
            fieldNamesArray = this.getFieldNames(false);

        for (var i = 0; i < fieldNamesArray.length; i++) {
        	
        	var fieldName = fieldNamesArray.get(i),
        	    value = original[fieldName],
        	    field = this.getField(fieldName);

            if (isc.isA.Date(value)) {
                var dateCopy = new Date();

                dateCopy.setTime(value.getTime());
                dateCopy.logicalDate = value.logicalDate;
                dateCopy.logicalTime = value.logicalTime;

                copy[fieldName] = dateCopy;

            // If the multiple flag is true and the field type is not
            // datasource then we want to duplicate the array if it is one.
            } else if (isc.isAn.Array(value) && 
            		   field.multiple == true && 
            		   field.type == null) 
            {
            	var arrayCopy = [];

            	for (var j =0; j < value.length; j++){

            		if (isc.isA.Date(value[j])) {
                        var dateCopy = new Date();

                        dateCopy.setTime(value[j].getTime());
                        dateCopy.logicalDate = value[j].logicalDate;
                        dateCopy.logicalTime = value[j].logicalTime;

                        arrayCopy[j] = dateCopy;
                    } else {
                    	arrayCopy[j] = value[j];
            		}
            	}

            	copy[fieldName] = arrayCopy;
            } else {
            	copy[fieldName] = value;
            }
        }
        return copy;
    },

    //> @method dataSource.copyRecords()
    // Copies all DataSource field values of an (Array) of Records (including a TreeNode) 
    // to a new array of Records, omitting component-specific metadata such as selected state 
    // from grids, or parent folders for TreeNodes. This method calls 
    // +link{DataSource.copyRecord()} for each item in the array.
    // @param records (Array of Record) The array of Record objects to be copied. 
    // @return (Array of Record) A new copy of each record provided in the array argument, 
    // with component-specific metata data removed.
    // @visibility external
    //<
    copyRecords : function (originals) {
    	if (originals == null) return null;

        var copies = [];

        for (var i = 0; i < originals.length; i++) {
        	var original = originals[i],
        	    copy = this.copyRecord(original);

        	copies[i] = copy;
        }
        return copies;
    },

    //> @method dataSource.transformRequest() [A]
    // For a dataSource using +link{group:clientDataIntegration,client-side data integration},
    // return the data that should be sent to the +link{dataURL}.
    // <P>
    // By default, HTTP requests sent to non-SmartClient servers do not include DSRequest
    // metadata such as +link{dsRequest.startRow}, +link{dsRequest.endRow,endRow},
    // and +link{dsRequest.oldValues,oldValues}.  Only the core
    // +link{group:dataSourceOperations,datasource protocol data} is sent, such as the criteria
    // passed to +link{listGrid.fetchData,fetchData()} or the updated values submitted by
    // +link{dynamicForm.saveData(),form.saveData()}.  
    // <P>
    // transformRequest() allows you to transform dsRequest metadata into a
    // format understood by your server and include it in the HTTP request, so that you can
    // integrate DataSource features such as data paging with servers that support such
    // features.
    // <P>
    // How the data is actually sent to the URL is controlled by 
    // +link{operationBinding.dataProtocol}.  If using the "getParams" or
    // "postParams" protocol, data is expected to be a JavaScript Object where each property
    // will become a GET or POST'd parameter.  If using dataProtocol:"soap" or "postXML", data
    // will be serialized as an XML message by +link{method:DataSource.xmlSerialize()}.
    // <P>
    // As an example, if you have a dataURL that can return paged data given URL parameters
    // "start" and "end", you could implement transformRequest like so:
    // <pre>
    //   isc.DataSource.create({
    //      ... 
    //      transformRequest : function (dsRequest) {
    //         if (dsRequest.operationType == "fetch") {
    //             var params = {
    //                start : dsRequest.startRow,
    //                end : dsRequest.endRow
    //             };
    //             // combine paging parameters with criteria
    //             return isc.addProperties({}, dsRequest.data, params);
    //         }
    //      }
    //   });
    // </pre>
    // Other reasons to implement transformRequest():
    // <ul>
    // <li> transform a +link{type:Criteria} object into the custom query language of a web
    //      service
    // <li> add a session id to requests that require authentication
    // <li> detect colliding updates by sending both updated values and the values the user
    //      originally retrieved before editing began (available as +link{dsRequest.oldValues})
    // </ul>
    // <i>Special case:</i> If the <code>dataProtocol</code> is <code>"clientCustom"</code>
    // the SmartClient system will not attempt to send data to the server in any way. Instead 
    // transformRequest should be implemented such that it accesses or updates the underlying
    // data-set and calls +link{DataSource.processResponse()} when the operation is complete. This
    // setting allows straightforward integration with non SmartClient comm mechanisms that 
    // directly send requests to the server (such as GWT-RPC), or handle data manipulation without
    // sending HTTP at all (such as Google Gears).
    // <P>
    // Note: The +link{RestDataSource} class overrides transformRequest() to handle xml-serializing
    // the request (including meta data) into a standard format.
    //
    // @param dsRequest (DSRequest) the DSRequest being processed
    // @return (any) data to be sent to the dataURL
    // @visibility xmlBinding
    //<
    transformRequest : function (dsRequest) {
        return dsRequest.data;
    },
    
    //> @method dataSource.getUpdatedData() [A]
    // Helper method to retrieve the updated data from a successful dataSource update or add 
    // operation.
    // @param dsRequest (DSRequest) Update request object passed to the server. Note that this
    //  request may have been modified by +link{DataSource.transformRequest()} 
    // @param dsResponse (DSResponse) Response returned by the server
    // @param useDataFromRequest (boolean) If +link{DSResponse.data} is empty, should data be
    //  derived from the submitted request.
    // @return data (DataSourceRecord || Array of DataSourceRecords} Updated data.  
    // @visibility external
    //<
    getUpdatedData : function (dsRequest, dsResponse, useDataFromRequest) {
        var updateData = dsResponse.data;
        // If the server failed to return the updated records, and updateCacheFromRequest is true,
        // integrate the submitted values into the cache if the operation was succesful.
        if (useDataFromRequest && 
            dsResponse.status == 0 && 
            (updateData == null || 
             (isc.isA.Array(updateData) && updateData.length == 0) || 
             isc.isAn.emptyString(updateData))
           ) 
        {
            this.logInfo("dsResponse for successful operation of type " + dsRequest.operationType + 
                         " did not return updated record[s]. Using submitted request data to update"
                         + " ResultSet cache.", "ResultSet");
            var requestData = dsRequest.data;
            if (dsRequest.data && isc.isAn.Object(dsRequest.data)) {
                
                // if operationType is an update, request data will be sparse so need to combine 
                // with oldValues
                if (dsRequest.operationType == "update") {
                    updateData = isc.addProperties({}, dsRequest.oldValues);
            
                    // Assertion - we only update one record at a time, so if submitted data is an array
                    // it will contain one entry that matches the 'oldValues'
                    if (isc.isAn.Array(requestData)) {
                        updateData = isc.addProperties(updateData, requestData[0]);  
                    } else {
                        updateData = isc.addProperties(updateData, requestData);
                    }
                    updateData = [updateData];
                // for add or delete old values are irrelevant
                } else {      
                    if (!isc.isAn.Array(requestData)) requestData = [requestData];
                    updateData = [];
                    for (var i = 0; i < requestData.length; i++) {
                        updateData[i] = isc.addProperties({}, requestData[i]);
                    }
                }
                //>DEBUG
                if (this.logIsDebugEnabled("ResultSet")) {
                    this.logDebug("Submitted data to be integrated into the cache:"
                                  + this.echoAll(updateData), "ResultSet");
                }
                //<DEBUG
            }
        }
        return updateData;
    },
    
    
    serializeFields : function (data, dsRequest) {
    

        if (!data && dsRequest != null) data = dsRequest.data;
        if (!data) return data;

        // don't try to serialize simpleType values
        if (isc.DS.isSimpleTypeValue(data)) return data;

        if (isc.isAn.Array(data)) {
            var newData = [];
            for (var i = 0; i < data.length; i++) {
                newData[i] = this.serializeFields(data[i], dsRequest);
            }
            return newData;
        } else if (this.isAdvancedCriteria(data)) {
            return this.serializeAdvancedCriteria(data);
        }
        data = isc.addProperties({}, data);

        
        if (data.__ref) {
            delete data.__ref;
            delete data.__module;
        }

        var fields = this.getFields();
        for (var fieldName in fields) {
        
            // Skip non-object values - we only care about dates
            // and nested objects which could contain dates.
            if (!isc.isAn.Object(data[fieldName])) continue;
        	
            var field = fields[fieldName],
                type = field.type,
                typeDS = type != null ? isc.DataSource.get(type) : null;
            	
            if (typeDS && typeDS.serializeFields) {
                data[fieldName] = typeDS.serializeFields(data[fieldName]);
            	
            } else if (isc.isA.Date(data[fieldName])) {
                if (isc.SimpleType.getBaseType(field.type) == "date" &&
                    !isc.SimpleType.inheritsFrom(field.type, "datetime"))
                {
                    // NOTE: we choose to flag the date instead of serializing it right here
                    // because we want isc.Comm.xmlSerialize() to use xsi:type="date" for clarity 
                    data[fieldName].logicalDate = true;
                } else if (isc.SimpleType.getBaseType(field.type) == "time") {
                    data[fieldName].logicalTime = true;
                }
            }
        }
        return data;
    },

    
    serializeAdvancedCriteria : function (data) {
        data = isc.clone(data);
        if (data.criteria) {
            for (var i = 0; i < data.criteria.length; i++) {
                data.criteria[i] = this.serializeAdvancedCriteria(data.criteria[i]);
            }
        } else {
            if (isc.isA.Date(data.value) || isc.isA.Date(data.start) || isc.isA.Date(data.end)) {
                var field = this.getField(data.fieldName);
                if (field != null) {
                    if (isc.SimpleType.getBaseType(field.type) == "date" &&
                        !isc.SimpleType.inheritsFrom(field.type, "datetime")) 
                    {
                        if (data.value) data.value.logicalDate = true;
                        if (data.start) data.start.logicalDate = true;
                        if (data.end) data.end.logicalDate = true;
                    } else if (isc.SimpleType.getBaseType(field.type) == "time") {
                        if (data.value) data.value.logicalTime = true;
                        if (data.start) data.start.logicalTime = true;
                        if (data.end) data.end.logicalTime = true;
                    }
                }
            }
        }
        return data;
    },
    
    
    //> @method DataSource.getDataProtocol() [A]
    // Returns the appropriate +link{OperationBinding.dataProtocol} for a +link{DSRequest}
    // @param dsRequest (DSRequest) DataSource Request object
    // @return (DSProtocol) DataProtocol to be used for this request operation.
    // @visibility external
    //<
    getDataProtocol : function (dsRequest) {
        var operationBinding = this.getOperationBinding(dsRequest),
            service = this.getWebService(dsRequest);
        // WSDL-described web services use SOAP unless a dataProtocol has been explicitly set
        // for the operationBinding.
        // NOTE: protocol per operationBinding allows eg GET fetch, POST update
        return (operationBinding.dataProtocol != null ? operationBinding.dataProtocol :
                isc.isA.WebService(service) ? "soap" : this.dataProtocol || "getParams");
    },

    _storeCustomRequest : function (dsRequest) {
        // clientCustom dataProtocol 
        // - transformRequest has been implemented to call non SmartClient code to generate the
        //   appropriate response, and call 'processResponse()' with it.
        // - store out the request so we can retrieve it by ID when processResponse fires.
        //   Doing this before calling transformRequest should handle synchronous as well as asynch
        //   response generation code.
        // - note we do this unconditionally since it is allowed to switch a request
        //   dynamically to clientCustom in transformRequest.  If it turns out it's not a
        //   clientCustom request, we just remove the entry later
        if (!this._clientCustomRequests) this._clientCustomRequests = {};
        this._clientCustomRequests[dsRequest.requestId] = dsRequest;
    },
    
    // get rpcRequest properties that should be set on this dsRequest based on dataProtocol and
    // other DataSource settings.
    // NOTE: 
    // - these settings will be applied to an rpcRequest which is a *copy* of the dsRequest,
    //   which is why we don't simply change the settings directly here.  See 
    //   "RPCRequest vs DSRequest" comment elsewhere in this file
    // - we need to be careful to respect overrides set on the dsRequest
    getServiceInputs : function (dsRequest) {
        
        var operationBinding = this.getOperationBinding(dsRequest),
            service = this.getWebService(dsRequest),
            operation = this.getWSOperation(dsRequest);

        this.addDefaultCriteria(dsRequest, operationBinding);

        // call transformRequest to allow the data to be changed before it is serialized to the
        // wire.  Hang onto the data in it's original format too
        dsRequest.originalData = dsRequest.data;

        this._storeCustomRequest(dsRequest);

        this.applySendExtraFields(dsRequest);

        // offline handling may mean we never send this request
        if (!this.clientOnly) {
            if (this.fulfilledFromOffline(dsRequest)) return { dataProtocol:"clientCustom" };

            // Save a copy of the unconverted DSRequest onto itself, for later use as a key in the
            // the isc.Offline cache.  See the comments near the call to convertRelativeDates() in 
            // DataSource.sendDSRequest() for details of why we do this.
            dsRequest.unconvertedDSRequest = isc.shallowClone(dsRequest);
        }

        if (this.autoConvertRelativeDates == true) {
            // convert any relative dates in criteria into absolute dates so the server
            // doesn't need to know how to handle relative dates
            if (this.logIsInfoEnabled("relativeDates")) {
                this.logInfo("Calling convertRelativeDates from getServiceInputs "+
                    "- data is\n\n"+isc.echoFull(transformedData));
            }

            var data = this.convertRelativeDates(dsRequest.data);

            if (this.logIsInfoEnabled("relativeDates")) {
                this.logInfo("Called convertRelativeDates from getServiceInputs "+
                    "- data is\n\n"+isc.echoFull(transformedData));
            }
            dsRequest.data = data;
        }

        var transformedData = this.transformRequest(dsRequest);
        // correct the common error of returning the dsRequest itself incorrectly, which is
        // never right since the dsRequest contains various widgets and other data
        // inappropriate to send to the server.
        if (transformedData !== dsRequest) {
            dsRequest.data = transformedData;
        }

        // If it was a clientCustom request, just return and wait for a call to
        // processResponse.  Note dynamic checks allows switching dataProtocol in
        // transformRequest
        var protocol = this.getDataProtocol(dsRequest),
            clientCustom = protocol == "clientCustom";
        if (clientCustom) {
            return {dataProtocol:"clientCustom"};
        } else {
            delete this._clientCustomRequests[dsRequest.requestId];
        }

        if (isc.isA.WebService(service)) {
            if (dsRequest.wsOperation == null && isc.isAn.Object(operation)) {
                dsRequest.wsOperation = operation.name;
            }
            this.logInfo("web service: " + service + 
                          ", wsOperation: " + this.echoLeaf(operation), "xmlBinding");
        }

        dsRequest._sentTime = isc.timestamp();
        
        var dataURL = this.getDataURL(dsRequest);

        // allow dataURL to be specified as a DSRequest property.  Needed because the same
        // structure of data may come from many different URLs (such as RSS feeds or WSDL
        // files).  Also allow actionURL because we advertise that all RPCRequest props are
        // legal on DSRequests.
        dataURL = dsRequest.actionURL || dsRequest.dataURL || dataURL;
 
        // allow useHttpProxy to be set on operationBinding or DataSource
        if (dsRequest.useHttpProxy == null) {
            dsRequest.useHttpProxy = this._firstNonNull(operationBinding.useHttpProxy,
                                                        this.useHttpProxy);
        }
       
        // getParams / postParams send the request data to the server as params.
        // In addition to this, regardless of data protocol we support explicit
        // per request params, and defaultParams from the DS / binding.
        var params,
            defaultParams = operationBinding.defaultParams || this.defaultParams,
            explicitParams = dsRequest.params;
        
        if (defaultParams || explicitParams) {
            params = isc.addProperties({}, defaultParams, explicitParams);
        }

        var dataAsParams = protocol == "getParams" || protocol == "postParams"; 
        if (dataAsParams) {
            params = isc.addProperties(params || {}, dsRequest.data);
        }
        
        if (dataAsParams) {
    
            if (params) params = this.serializeFields(params, dsRequest);
            
            // GET or POST with form-style params
            return {
                actionURL: dataURL,
                httpMethod: dsRequest.httpMethod || (protocol == "getParams" ? "GET" : "POST"),
                params: params
            };
        }
        
        // remaining possibilities ("postMessage", "postXML", "soap") all POST a request body

        var serviceInputs = {
            actionURL: dataURL,
            // allow httpMethods like "PUT"
            httpMethod: dsRequest.httpMethod || "POST"
        };
        // if necessary pass params to the server as well as data in the body
        if (params) serviceInputs.params = params;
        
        if (protocol == "postMessage") {
            serviceInputs.data = (dsRequest.data || "").toString();
        }

        // for XML posts, serialize the data as XML
        if (protocol == "postXML" || protocol == "soap") {
            var flags = this.getSerializeFlags(dsRequest);
            var requestBody = serviceInputs.data = this.getXMLRequestBody(dsRequest);
            serviceInputs.contentType = dsRequest.contentType || "text/xml";
            this.logDebug("XML post requestBody is: " + requestBody, "xmlBinding");
        }

        // add SOAPAction HTTP header for SOAP
        if (protocol == "soap") {
            var soapAction = this._firstNonNull(operationBinding.soapAction, 
                                                operation.soapAction);
            // NOTE: some engines, like Axis, require a soapAction to be present, so people who
            // don't really need a soapAction put soapAction="" into their WSDL, hoping for a
            // meaningless 'SOAPAction: ""' header to get by Axis.
            if (soapAction == null) soapAction = '""';
            serviceInputs.httpHeaders = isc.addProperties({ SOAPAction : soapAction },
                                                            dsRequest.httpHeaders);

            var spoofResponses = isc.isA.WebService(service) &&
                                 this._firstNonNull(operationBinding.spoofResponses,
                                                    this.spoofResponses);
                                
            if (spoofResponses) {
                serviceInputs.spoofedResponse = service.getSampleResponse(operation.name);
                this.logInfo("Using spoofed response:\n" + serviceInputs.spoofedResponse, "xmlBinding");
            }
            
        }

        if (this.logIsDebugEnabled("xmlBinding")) {
            this.logDebug("serviceInputs are: " + this.echo(serviceInputs), "xmlBinding");
        }
        return serviceInputs;
    },
    
    addDefaultCriteria : function (dsRequest, operationBinding) {
        var defaultCriteria = operationBinding.defaultCriteria || this.defaultCriteria;
        
        // nothing to do if there's no defaultCriteria
        if (!defaultCriteria) return;
        
        defaultCriteria = isc.addProperties({}, defaultCriteria);
        if (defaultCriteria && dsRequest.operationType == "fetch") {
            if (this.isAdvancedCriteria(dsRequest.data)) {
                // combineCriteria will retain references to like fields from both criteria sets,
                // which is inappropriate for this use case - we want the real values to hide the
                // default values, otherwise we'll end up with something like 
                // "myField = 'defaultValue' AND myField = 'realValue'".  This isn't straightforward
                // because AdvancedCriteria can be arbitrarily complex.  So, we'll assume that the 
                // defaultCriteria is always simple (and document it that way) and remove from it
                // any property that is named anywhere in the existing criteria
                var fieldNames = this.mineCriteriaFieldNames(dsRequest.data);
                for (var key in defaultCriteria) {
                    if (fieldNames.contains(key)) delete defaultCriteria[key];
                }
            }
            // nothing to do if all defaultCriteria were removed (because they're already used!)
            if (isc.getValues(defaultCriteria).length > 0) {
                dsRequest.data = isc.DataSource.combineCriteria(dsRequest.data, 
                                    defaultCriteria, "and", null);
            }
        }
    },
    
    mineCriteriaFieldNames : function (criteria) {
    	var fieldNames = [];
        if (!criteria.criteria) return fieldNames;
        
        for (var i = 0; i < criteria.criteria.length; i++) {
            if (criteria.criteria[i].criteria) {
                fieldNames.addList(this.mineCriteriaFieldNames(criteria.criteria[i]));
            } else {
                fieldNames.add(criteria.criteria[i].fieldName);
            }
        }
        return fieldNames;
    },
    
    applySendExtraFields : function (dsRequest) {
        // If sendExtraFields is false, remove any non-ds fields from the record(s) in request.data
        // before calling transformRequest
                
        if (!this.sendExtraFields) {
            var data = dsRequest.data;
            if (!isc.isAn.Array(data)) data = [data];
            for (var i = 0; i < data.length; i++) {
                var record = data[i];
                if (!isc.isAn.Object(record)) continue;
                
                for (var field in record) {                    
                    if (!this.getField(field)) delete record[field];
                }
            }
        }
    },
    
    //> @method DataSource.processResponse() (A)
    // Process a dsResponse for a request initiated by a DataSource with
    // +link{OperationBinding.dataProtocol,dataProtocol:"clientCustom"}.
    // <code>requestId</code> parameter should be dsRequest.requestId as found on the dsRequest
    // passed to +link{dataSource.transformRequest()}.
    // <P>
    // You must provide a response for both error and non-error cases.  For an error case, a
    // sufficient response is:
    // <pre>
    // { status : -1 }
    // </pre>
    // @param requestId (string) requestId attribute from the associated dataSource request object
    // @param dsResponse (DSResponse properties) Configuration for the dsResponse 
    // @visibility external
    //<
    processResponse : function (requestId, dsResponse) {
        var dsRequest = this._clientCustomRequests[requestId];
        if (dsRequest == null) {
            this.logWarn("DataSource.processResponse(): Unable to find request corresponding to ID "
                        + requestId + ", taking no action.");
            return;
        }
        delete this._clientCustomRequests[requestId];
        
        // assume success
        if (dsResponse.status == null) dsResponse.status = 0;
        if (dsResponse.status == 0) {
            // If start/endRow are unset check for data length
            var data = dsResponse.data;
            if (data == null) dsResponse.data = data = [];
            if (dsResponse.startRow == null) dsResponse.startRow = dsRequest.startRow || 0;
            if (dsResponse.endRow == null) dsResponse.endRow = dsResponse.startRow + data.length;
            if (dsResponse.totalRows == null) {
                dsResponse.totalRows = Math.max(dsResponse.endRow, data.length);
            }
        }

        this._completeResponseProcessing(data, dsResponse, dsRequest);
    },

    // An undocumented threshold over which the call to getClientOnlyResponse(), when
    // handling the data source reply of a client only data source, is to be executed
    // asynchronously.
    _asyncClientOnlyResponseThreshold: 200,

    _handleClientOnlyReply : function (rpcResponse, data, rpcRequest) {
        var serverData = this.cacheAllData && !this.clientOnly && this.cacheResultSet ? 
                this.cacheResultSet.getAllRows() : null,
            dsRequest = rpcRequest._dsRequest;

        // getClientOnlyResponse() is called asynchronously when the number of records is over
        // the configurable threshold _asyncClientOnlyResponseThreshold and copyLocalResults is
        // true.
        var callback = function (dsResponse) {
            // use by the Developer Console RPC tab to show clientOnly responses
            rpcResponse.clientOnlyData = dsResponse ? dsResponse.data : null;

            this._completeResponseProcessing(data, dsResponse, dsRequest, rpcResponse, rpcRequest);
        };
        if (this.copyLocalResults &&
            this.getClientOnlyResponse === isc.DataSource.getInstanceProperty("getClientOnlyResponse"))
        {
            this._asyncGetClientOnlyResponse(dsRequest, serverData, callback);
        } else {
            var dsResponse = this.getClientOnlyResponse(dsRequest, serverData);
            callback.call(this, dsResponse);
        }
    },

    
    _asyncGetClientOnlyResponse : function (request, serverData, callback) {
        //!OBFUSCATEOK

        var operationType = request.operationType,
            response = {
                status:0
            };

        if (!(operationType == "fetch" || operationType == "select" || operationType == "filter")) {
            // The synchronous version of getClientOnlyResponse() should be sufficient for handling
            // the other operation types (remove, delete, add, insert, replace, update, and validate).
            callback.apply(this, [this.getClientOnlyResponse(request, serverData)]);
            return;
        }

        // initialize the spoofed dataset

        serverData = serverData || this.testData;

        var cacheData = this.cacheData,
            testData = this.testData,
            setCacheDataAndTestDataToServerData = false;

        // if we have serverData but no testData, it's a cacheAllData result - if we're also in
        // clientOnly mode, we need to set up the clientOnly cache - set both cacheData and the
        // now deprecated testData to the same array
        if (serverData && !this.testData && this.clientOnly) {
            cacheData = testData = serverData;
            setCacheDataAndTestDataToServerData = true;
        }

        if (!serverData || isc.isA.String(serverData)) {
            if (isc.isA.String(serverData)) {
                // ID or expression
                //>DEBUG
                this.logInfo(this.ID + " datasource: using testData property as data");
                //<DEBUG
                cacheData = testData = isc.eval(serverData);
                setCacheDataAndTestDataToServerData = true;
            } else if (window[this.ID + "TestData"]) {
                // dataset loaded in page under canonical name
                //>DEBUG
                this.logInfo(this.ID + " datasource: using " + this.ID + "TestData object as data");
                //<DEBUG
                cacheData = testData = window[this.ID + "TestData"];
                setCacheDataAndTestDataToServerData = true;
            } else {
                // initialize to empty list
                //>DEBUG
                this.logInfo(this.ID + " datasource: testData property and " + this.ID +
                             "TestData object not found, using empty list as data");
                //<DEBUG
                cacheData = testData = [];
                setCacheDataAndTestDataToServerData = true;
            }
            serverData = testData;
        }

        var criteria = request.data;
        // support multiple where clause format (don't think this is actually valid any
        // more)
        if (isc.isAn.Array(criteria)) criteria = criteria[0];
        var filteredData = this.applyFilter(serverData, criteria, request),
            resultData = filteredData;

        if (request.startRow != null) {
            // simulate paging
            var startRow = request.startRow,
                endRow = request.endRow,
                totalRows = filteredData.length;

            // server handles sort, so sort the data by the requested direction
            
            var sortProperty = isc.shallowClone(request.sortBy);
            if (sortProperty) {
                if (!isc.isAn.Array(sortProperty)) sortProperty = [sortProperty];

                if (isc.isAn.Object(sortProperty[0])) {
                    // if this is an array or SortSpecifiers, map them to sortBy strings
                    sortProperty = isc.DS.getSortBy(sortProperty)
                }

                var sortDirs = [];

                for (var i = 0; i < sortProperty.length; i++) {
                    var sortDirection = true;
                    if (sortProperty[i].startsWith("-")) {
                        sortProperty[i] = sortProperty[i].substring(1);
                        sortDirection = false;
                    }
                    sortDirs[i] = sortDirection;
                }
                filteredData.sortByProperties(sortProperty, sortDirs);
            }

            // cap the endRow at one after last valid index (it's exclusive)
            endRow = Math.min(endRow, totalRows);
            // slice out from startRow to endRow (note: slice is non-inclusive at the end)
            resultData = filteredData.slice(startRow, endRow);
            response.startRow = startRow;
            response.endRow = endRow;
            response.totalRows = totalRows;
        }

        // shallow copy the results.  This allows test or example cases where the
        // "server data set" is changing independently of the client results
        var finish = function () {
            response.data = resultData;

            this._trackClientOnlyChanges(response, operationType);

            if (setCacheDataAndTestDataToServerData) {
                this.cacheData = this.testData = serverData;
            }

            callback.call(this, response);
        };
        if (this.copyLocalResults) {
            this._asyncCopyLocalResults(0, resultData.length, resultData, finish);
        } else {
            finish.call(this);
        }
    },

    _asyncCopyLocalResults : function (startIndex, endIndex, resultData, callback) {
        var finalIndex = endIndex,
            isSGWT = isc.Browser.isSGWT;
        endIndex = Math.min(endIndex, startIndex + this._asyncClientOnlyResponseThreshold);
        for (var i = startIndex; i < endIndex; ++i) {
        	if (this.deepCopyLocalResults) {
        		resultData[i] = isc.clone(resultData[i]);
			} else {
	            resultData[i] = isc.addProperties({}, resultData[i]);
			}
            // Clear out SGWT's __ref/__module pointers so that if the cacheData is created in
            // SGWT, Record.getOrCreateRef() won't return the same Java Record object for the
            // copied result record.
            if (isSGWT) {
                resultData[i][isc.gwtRef] = null;
                resultData[i][isc.gwtModule] = null;
            }
        }
        if (endIndex == finalIndex) {
            callback.call(this);
        } else {
            this.delayCall("_asyncCopyLocalResults", [endIndex, finalIndex, resultData, callback], 0);
        }
    },
    
    _handleCustomReply : function (rpcResponse, data, rpcRequest) {
        var dsResponse = {
            data : data,
            startRow : 0,
            endRow : 0,
            totalRows : 0,
            status : 0
        };
        var dsRequest = rpcRequest._dsRequest;
        this._completeResponseProcessing(data, dsResponse, dsRequest, rpcResponse, rpcRequest);
    },

    _handleJSONReply : function (rpcResponse, data, rpcRequest) {
        var dsRequest = rpcRequest._dsRequest,
            recordXPath = this.getOperationBinding(dsRequest).recordXPath || this.recordXPath;
        
        // log the raw object reply if we didn't just eval it from text (if we eval it from
        // text we've already logged the raw response)
        if ((rpcResponse._wasJSONTextReply || rpcResponse._wasCSVTextReply) && this.logIsDebugEnabled("xmlBinding")) {
            this.logDebug("Raw response data: " + this.echoFull(data), "xmlBinding");
        }

        // Hang onto the raw JSON object returned by the server. This allows transformResponse
        // to actually do something useful with the data returned (EG, pick up meta-data)
        var rawData = data;
        
        // Don't try to process null data - this can now happen if we get an invalid 
        // HTTP response (404, 500, etc) and willHandleError == true
        if ( data ) {
	        if (recordXPath) {
	            // extract relevant records via XPath
	            data = isc.xml.selectObjects(data, recordXPath);
	            this.logInfo("JSON recordXPath: '" + recordXPath + 
	                         "', selected: " + this.echoLeaf(data), "xmlBinding");
	        }
	
	        data = this.recordsFromObjects(data);
	
	        if (this.logIsDebugEnabled("xmlBinding")) {
	            this.logDebug("Validated dsResponse.data: " + isc.echoFull(data), "xmlBinding");
	        }
	
	        // derive the default response
	        var dsResponse = {
	            data : data,
	            startRow : dsRequest.startRow || 0, 
	            status : 0 // assume success
	        };       

	        dsResponse.endRow = dsResponse.startRow + Math.max(0, data.length);
	        dsResponse.totalRows = Math.max(dsResponse.endRow, data.length);
	        
        } else {
	        // derive the default error response
            var status = rpcResponse.status;
            if (status == 0 || status == null) status = -1;
	        var dsResponse = {
	            status : status,
                data:rpcResponse.data
	        };       
        }

        this._completeResponseProcessing(rawData, dsResponse, dsRequest, rpcResponse, rpcRequest);
    },
    
    _handleCSVTextReply : function (rpcResponse, csvText, rpcRequest) {
        if (rpcResponse.status != 0) return;

        var settings = isc.TextImportSettings.create({
            hasHeaderLine: true,
            trim: true
        });
        isc.TextImportSettings._importAsJsonObjects = true;
        var jsonObjects = this.recordsFromText(rpcResponse.data, settings);
        isc.TextImportSettings._importAsJsonObjects = false;        

        rpcResponse._wasCSVTextReply = true; 
        this._handleJSONReply(rpcResponse, jsonObjects, rpcRequest);
    },    

    _handleJSONTextReply : function (rpcResponse, jsonText, rpcRequest) {
        //!DONTOBFUSCATE
        
        // If the original request came from RestDataSource and formed part of a multi-op 
        // queue, queue-handling code in RPCManager will have eval'd the response already, 
        // so what is passed in here as "jsonText" is actually a real JS object
        if (rpcRequest.isRestRequest && !isc.isA.String(jsonText)) {
            var jsonObjects = jsonText;
        } else {
            // Don't try to process jsonText if we didn't get a valid response
            if (rpcResponse.status >= 0) {
                var evalText = jsonText;
                // Strip the json prefix / suffix if supplied, and transport was not scriptInclude
                if (rpcRequest.transport != "scriptInclude" && evalText != null) {
                    // Strip whitespace before checking for specified prefix / suffix
                    var re;
                    if (this.jsonPrefix) {
                        re = new RegExp(/^\s*/);
                        evalText = evalText.replace(re, "");
                        if (evalText.startsWith(this.jsonPrefix)) {
                            evalText = evalText.substring(this.jsonPrefix.length);
                        } else {
                            this.logInfo("DataSource specifies jsonPrefix, but not present in " + 
                                         "response returned from server. Processing response anyway.");
                        }
                    } 
                    if (this.jsonSuffix) {
                        re = new RegExp(/\s*$/)                
                        evalText = evalText.replace(re, "");            
                        if (evalText.endsWith(this.jsonSuffix)) {
                            evalText = evalText.substring(0, (evalText.length - this.jsonSuffix.length));
                        } else {
                            this.logInfo("DataSource specifies jsonSuffix, but not present in " + 
                                         "response returned from server. Processing response anyway.");
                        }
                    }
                }

                try {
                	var jsonObjects;
                	if (rpcRequest.useStrictJSON) {
                		jsonObjects = isc.Class.parseStrictJSON(evalText, rpcRequest.jsonReviver);
					} else {
						// Note: FF1.0.7 bug: curly must be escaped in the following regex.  Not an issue in IE
						// or FF1.5
						if (evalText && evalText.match(/^\s*\{/)) {
							// eval barfs on "{ ...  }", thinking it's a closure
							evalText = "var evalText = " + evalText + ";evalText;"
						}
	                    jsonObjects = isc.eval(evalText);
					}
                } catch (e) {
                	
                    rpcResponse.status = -1;
                    rpcResponse.data = "Error: server returned invalid JSON response";
                    this.logWarn("Error evaluating JSON: " + e.toString() + ", JSON text:\r" + jsonText);
                }

                // warn and bail if server response object evals to null.
                // This can happen if the response is "", or if it's something like "var foo = {};"
                // where the result of the eval is undef.
                // Avoids a JS error in handleJSONReply
                if (jsonObjects == null) {
                    rpcResponse.status = -1;
                    rpcResponse.data = "Error: server returned invalid JSON response";
                    this.logWarn("Evaluating JSON reply resulted in empty value. JSON text:\r" + this.echo(jsonText));
                    jsonObjects = jsonText;
                }
        
                // Hack to let _handleJSONReply know this JSON response came from text, for logging
                // purposes
                rpcResponse._wasJSONTextReply = true; 
            }
        }
        
        this._handleJSONReply(rpcResponse, jsonObjects, rpcRequest);
    },
    
    //> @method dataSource.recordsFromObjects() [A]
    // Transform a list of Objects to DataSource records.
    // <P>
    // The value for each field is extracted from the XML according to the
    // rules described under +link{attr:DataSourceField.valueXPath}.  
    //<
    recordsFromObjects : function (data) {

        // normalize to Array and apply schema
        if (!isc.isAn.Array(data)) data = [data];

        // skipping validation means eg date values specified as Strings won't become Dates,
        // valueXPath and dropExtraFields won't apply, etc.  But for large data volumes where
        // these features aren't required, this is faster.  Undocumented for now.
        if (this.skipJSONValidation) return data;

        for (var i = 0; i < data.length; i++) {
            data[i] = this.validateJSONRecord(data[i]);
        }
        return data;
    },

    validateJSONRecord : function (record, disableXPath) {
        var fieldNames = this.getFieldNames(),
            result = {};
        for (var i = 0; i < fieldNames.length; i++) {
            var fieldName = fieldNames[i],
                field = this.getField(fieldName),
                fieldValue;

            if (field.valueXPath && !disableXPath) {
                fieldValue = isc.xml.selectObjects(record, field.valueXPath, true);
            } else {
                fieldValue = record[fieldName];
            }
            
            if (field.getFieldValue) {
                if (!isc.isA.Function(field.getFieldValue)) {
            	    isc.Func.replaceWithMethod(field, "getFieldValue", 
                                                     "record,value,field,fieldName");
                }
                fieldValue = field.getFieldValue(record, fieldValue, field, fieldName);
            }

            var undef;
            if (fieldValue !== undef) {
                // validation fieldValue, if it is complex type
                var fieldDS = isc.DS.get(field.type);
                if (fieldDS && !(fieldDS.skipJSONValidation)) {
                    if (!(isc.isAn.Array(fieldValue))) {
                        fieldValue = fieldDS.validateJSONRecord(fieldValue);    
                    } else {
                        for (var j = 0; j < fieldValue.length; j++) {
                            fieldValue[j] = fieldDS.validateJSONRecord(fieldValue[j]);
                        }
                    }                    
                }

                //this.logWarn("validating value: " + fieldValue + 
                //             " of field: " + this.echo(field));
                result[fieldName] = this.validateFieldValue(field, fieldValue);
            }
        }

        if (this.dropExtraFields) return result;
            
        
        for (var i = 0; i < fieldNames.length; i++) {
            var fieldName = fieldNames[i];
            record[fieldName] = result[fieldName]
        }
        return record;
    },
		
    // XML Request Formation
    // ---------------------------------------------------------------------------------------

    // get the schema to be used to serialize JS data to XML.  
    // Also takes a dsRequest.
    getMessageSerializer : function (operationType, operationId) {
        // either: 
        // - an operationBinding has been specified with a wsOperation, so we use that, or
        // - no operationBinding was specified, so getOperationBinding() returns this, and then:
        //   - if this.wsOperation has been set, we use it
        //   - otherwise, we don't seem to be connected to a web service, so we use this
        //     DataSource as the message schema
        var operationBinding = this.getOperationBinding(operationType, operationId);
        if (operationBinding.wsOperation) {
            var service = this.getWebService(operationType, operationId);
            return service.getMessageSerializer(operationBinding.wsOperation);
        }
        return this;
    },

    //> @method dataSource.getXMLRequestBody() [A]
    // Get the XML to be posted to the dataURL based on the passed DSRequest.
    // <P>
    // This API is intended to be overridden in order to integrate with web services that
    // expect XML messages rather than simple HTTP parameters, but lack a
    // WSDL description.  For WSDL-described web services, having loaded the service
    // description, SmartClient knows the correct XML message structure, so customization is
    // best done by modifying the JavaScript data that is used to form the message.
    //
    // @see XMLTools.loadWSDL() for integration with WSDL-described web services
    // 
    // @param dsRequest (DSRequest) the request to encode as an XML message.
    // @return (String) the entire XML request as a String, including SOAP envelope if SOAP is used
    // @visibility xmlBinding
    //<
    getXMLRequestBody : function (dsRequest, flags) {
        if (isc._traceMarkers) arguments.__this = this;

        var requestBody = isc.SB.create(),
            protocol = this.getDataProtocol(dsRequest);

        // start envelope and headers
        if (protocol == "soap") {
            requestBody.append(this.getSoapStart(dsRequest), "\r");
            requestBody.append(this.getSoapBody(dsRequest, flags));
            requestBody.append("\r", this.getSoapEnd(dsRequest));
        } else { 
            if (this.messageStyle == "template") {
                requestBody.append(this._createTemplatedRequestBody(dsRequest));
            } else {
                var schema = this.getMessageSerializer(dsRequest);
                var flags = this.getSerializeFlags(dsRequest, flags);
                requestBody.append(schema.xmlSerialize(dsRequest.data, flags));
            }
        }

        if (this.logIsDebugEnabled("xmlComm")) {
            this.logDebug("outbound XML message: " + requestBody, "xmlComm");
        }

        return requestBody.toString();
    },

    
    _createTemplatedRequestBody : function (dsRequest) {
        var requestBody = isc.SB.create(),
            template = this.soapBodyTemplate,
            message;

        
            message = template.evalDynamicString(this, dsRequest);
        

        return message;
    },

    // WebService-related helper methods
    getSchemaSet : function () {
        return isc.SchemaSet.get(this.schemaNamespace);
    },

    hasWSDLService : function (dsRequest) {
        return isc.isA.WebService(this.getWebService(dsRequest));
    },

    getWebService : function (dsRequest) {
        var operationBinding = this.getOperationBinding(dsRequest),
            serviceNamespace = (dsRequest ? dsRequest.serviceNamespace : null) 
                                    || operationBinding.serviceNamespace || this.serviceNamespace,
            serviceName = (dsRequest ? dsRequest.serviceName : null) 
                                    || operationBinding.serviceName || this.serviceName;

        // see if we have a WebService instance with this serviceName / serviceNamespace
        var service;
        if (serviceName) service = isc.WebService.getByName(serviceName, serviceNamespace);
        else service = isc.WebService.get(serviceNamespace);

        if ((serviceNamespace != null || serviceName != null) && service == null) {
            this.logWarn("Could not find WebService definition: " +
                         (serviceName ? "serviceName: " + serviceName : "") +
                         (serviceNamespace ? "   serviceNamespace: " + serviceNamespace : "")
                         + this.getStackTrace());
        }

        // if we don't, then the DataSource itself is expected to have all the relevant
        // properties
        return service || this;
    },
    
    getWSOperation : function (dsRequest) {
        
        var operationBinding = this.getOperationBinding(dsRequest),
            wsOperation = (isc.isAn.Object(dsRequest) ? dsRequest.wsOperation : null) || 
                            operationBinding.wsOperation || this.wsOperation,
            service = this.getWebService(dsRequest);

        if (wsOperation != null && isc.isA.WebService(service)) {
            var returnOp = service.getOperation(wsOperation);
            if (!returnOp) {
                isc.logWarn("DataSource.getWSOperation() : could not retrieve the operation " 
                    + wsOperation);
            }
            return returnOp;
        }
        return this;
    },

    getDataURL : function (dsRequest) {
        var operationBinding = this.getOperationBinding(dsRequest);

        // if we have a real operationBinding definition (we're not using "this" as the
        // operationBinding definition) and it has a dataURL set, use that.  This allows eg
        // fetch via GET from one URL, save via POST to another
        if (operationBinding != this && operationBinding.dataURL) return operationBinding.dataURL;

        // NOTE: allow dataURL to be specified even on a DataSource that is using a web
        // service.  This is a simple way to spoof web service responses for a single
        // DataSource; using webService.setLocation() allows you to put the spoofing at the
        // WebService level
        if (this.dataURL != null) return this.dataURL;

        // otherwise if we have a WSDL-derived WebService definition, ask it for a dataURL
        // (WSDL services can't really have multiple URLs, but it's useful to set a different
        // URL per operation for spoofing during testing)
        if (this.hasWSDLService(dsRequest)) {
            var service = this.getWebService(dsRequest);
            return service.getDataURL(this.getWSOperation(dsRequest).name);
        }
        
        return this.dataURL;
    },

    //> @attr dataSource.globalNamespaces (Object : ... : IRW)
    // Namespaces definitions to add to the root element of outbound XML messages sent to a web
    // service, as a mapping from namespace prefix to namespace URI.
    // <P>
    // The default value is:
    // <pre>
    //   globalNamespaces : {
    //      xsi: "http://www.w3.org/2001/XMLSchema-instance",
    //      xsd: "http://www.w3.org/2001/XMLSchema"
    //   },
    // </pre>
    // This default value allows the use of the xsi:type and xsi:nil attributes without further
    // declarations.
    // <P>
    // Note that some web services will only accept specific revisions of the XML Schema URI.
    // If xsi-namespaced attributes seem to be ignored by an older webservice, try the URI
    // "http://www.w3.org/1999/XMLSchema-instance" instead.
    //
    // @serverDS allowed
    // @visibility xmlBinding
    //<
    getGlobalNamespaces : function (dsRequest) {
        var service = this.getWebService(dsRequest),
            namespaces = this.globalNamespaces;
        if (service && service.globalNamespaces) {
            namespaces = isc.addProperties({}, namespaces, service.globalNamespaces);
        }
        return namespaces;
    },

    getSoapStart : function (dsRequest) {
        // allow centralized overrides of getSoapStart/End via overrides on a WebService object
        var service = this.getWebService(dsRequest);
        if (service.getSoapStart) return service.getSoapStart(dsRequest);

        return isc.SB.concat(isc.DataSource._soapEnvelopeStart, 
                             isc.xml._generateNamespaces(this.getGlobalNamespaces(dsRequest), 
                                                         null, "         "),
                             ">", isc.DataSource._soapHeaderStart,
                             this.getSoapHeader(dsRequest),
                             isc.DataSource._soapHeaderEnd);
    },
    getSoapHeader : function (dsRequest) { 
        var service = this.getWebService(dsRequest);
        if (service.getSoapHeader) return service.getSoapHeader(dsRequest);

        // take data from either the request or from the service
        var headerData = dsRequest.headerData || service.getHeaderData(dsRequest);

        if (!headerData) return;

        this.logDebug("headerData is: " + this.echo(headerData), "xmlBinding");

        // get schema for the headers
        var allHeaderSchema = service.getInputHeaderSchema(this.getWSOperation(dsRequest)) || 
                                isc.emptyObject;
 
        // WSDL/SOAP blows: in SOAP the <wsdl:part>s of a <wsdl:message> do not have any name or
        // means of referring to them other than by the combination of the wsdl:message@name and
        // wsdl:part@name attributes.  When asking developers to specify data to be serialized
        // for each part, we rely on just the part name.  It would be perverse to declare a
        // soap header or body that uses two identically named parts of different messages, but
        // it appears to be legal. 
        // Relevant spec: http://www.w3.org/TR/wsdl#_soap:header
        var output = "", flatHeaders = dsRequest.useFlatHeaderFields;
        for (var partName in headerData) {
            var headerSchema = allHeaderSchema[partName];
            if (headerSchema != null) {
                //this.logWarn("serialized partName: " + partName + 
                //             " with schema " + headerSchema);
                if (isc.isA.DataSource(headerSchema)) {
                    output += headerSchema.xmlSerialize(headerData[partName], 
                                                        { useFlatFields: flatHeaders })
                } else {
                    // special handling for simpleType headers:
                    // The wsdl:message declaration, which becomes a DataSource, does not have
                    // a namespace (it doesn't belong to an xs:schema), however the wsdl:parts
                    // refer to types declared in an xs:schema block that must be namespaced.
                    // When a wsdl:part refers to a simpleType element, there won't be a
                    // DataSource for the element, only a field, hence the field has to carry
                    // the namespace via the special partNamespace property.
                    // headerSchema, in this case, is a field definition.
                    output += "\r     " + this._serializeSimpleTypeTag(partName, headerSchema,
                                    headerData[partName], headerSchema.partNamespace);
                }
            } else {
                this.logWarn("headerData passed for SOAP header partName: " + partName +
                             ", no schema available, not outputting");
            }
        }
        return output;
    },
    getSoapBody : function (dsRequest, flags) {
        if (isc._traceMarkers) arguments.__this = this;

        var soapBody = isc.SB.create(),
            service = this.getWebService(dsRequest),
            soapStyle = this.getSoapStyle(dsRequest),
            wsOperation = this.getWSOperation(dsRequest),

            // pass in a "flags" object that already has nsPrefixes, and reuse this flags
            // object to serialize each soap:body part.  This will capture all required
            // namespaces so we can output them as part of the soap:body element.   
            // For readability of the SOAP message, include the explicitly specified 
            // namespace prefixes.
            userPrefixes = this.xmlNamespaces ? isc.makeReverseMap(this.xmlNamespaces) : null,
            flags = isc.addProperties({
                nsPrefixes : isc.addProperties({}, userPrefixes)
            }, flags),

            // A soap:body has exactly one wsdl:message, represented by a DataSource, where each
            // DataSource field represents a wsdl:part of the message.  
            messageSchema = flags.generateResponse ? service.getResponseMessage(wsOperation.name)
                                                   : service.getRequestMessage(wsOperation.name),
            // 1. the soap:body can have multiple parts, either because:
            // - soap:body@parts is unspecified and the message has multiple parts
            // - soap:body@parts is specified and is a list
            // 2. each part can be just a simple type element or a complex element
            // 
            // Relevant spec: http://www.w3.org/TR/wsdl#_soap:body
            // - NOTE: not explicitly covered in the spec above, but based on WSDL4J APIs and
            //   other samples, soap:body@parts can be a space-separated *list* of part names
            //
            // Contrast this with soap:headers:
            // 1. each soap:header can have a different message
            // 2. each soap:header must specify a part
            bodyPartNames = flags.bodyPartNames || 
                                service.getBodyPartNames(wsOperation.name, flags.generateResponse);

        flags = this.getSerializeFlags(dsRequest, flags);

        
        isc.Comm.omitXSI = wsOperation.inputEncoding != "encoded";
        var oldXMLSchemaMode = isc.Comm.xmlSchemaMode;
        isc.Comm.xmlSchemaMode = true;

        var indent = "        ";

        if (soapStyle == "rpc") {
            // for RPC-style SOAP, SOAP body's outermost element should be named after the
            // operation and namespaced with the operation namespace 
            soapBody.append(
                "\n", indent, 
                isc.Comm._xmlOpenTag(wsOperation.name, null, 
                                     wsOperation.inputNamespace, "opNS", true), 
                ">"
            );
            indent += "    ";
            // otherwise the message definition as such does not produce outputs, only it's
            // parts do 
        }

        this.logInfo("soap:body parts in use: '" + bodyPartNames +
                     "', soapStyle: " + soapStyle, "xmlSerialize");
        if (this.logIsDebugEnabled("xmlSerialize")) {
            this.logDebug("SOAP data is: " + this.echoFull(dsRequest.data), "xmlSerialize");
        }

        for (var i = 0; i < bodyPartNames.length; i++) {
            var partName = bodyPartNames[i];

            // for document-style SOAP, if there is exactly one message part (the most common
            // style by far), use the entirety of the request data for that part.
            // This means that when a message is supposed to look like this:
            //    <login>
            //       <username>bob</username>
            //       <password>mebob</password>
            //    </login>
            // The JS data you need to pass is: 
            //    { username:"bob", password:"mebob" }
            // .. instead of the surprising and less obvious:
            //    { login : { username:"bob", password:"mebob" } }
            // Note that this works in conjunction with webService.getInputDS(), which for a
            // single part message will return the schema of that single part, such that a form
            // bound to that schema will produce the appropriate values to pass as
            // dsRequest.data.
            var data = bodyPartNames.length < 2 && soapStyle == "document" ? 
                            dsRequest.data : (dsRequest.data ? dsRequest.data[partName] : null);

            var partXML = messageSchema.getMessagePart(partName, data, flags, indent);
    
            soapBody.append("\r" + indent + partXML);
        }

        if (soapStyle == "rpc") {
            soapBody.append(
                "\n", "        ", isc.Comm._xmlCloseTag(wsOperation.name,
                                                        wsOperation.inputNamespace, "opNS")
            );
        }

        // restore global XML serialization settings
        isc.Comm.omitXSI = null;
        isc.Comm.xmlSchemaMode = oldXMLSchemaMode;

        return isc.SB.concat("    ", isc.DS._soapBodyStart,
                             this.outputNSPrefixes(flags.nsPrefixes, "        "),
                             ">",
                             soapBody.toString(),
                             "\r    ", isc.DS._soapBodyEnd);
    },

    // serialize a <wsdl:part> of a <wsdl:message>
    getMessagePart : function (partName, data, flags, indent) {
        if (isc._traceMarkers) arguments.__this = this;

        var partField = this.getPartField(partName),
            partSchema = this.getSchema(partField.type, partField.xsElementRef ? "element" : null),
            shouldLog = this.logIsInfoEnabled("xmlSerialize");

        // As with DataSource fields in general, each wsdl:part of a wsdl:message can be simple
        // or complex type
        if (isc.isA.DataSource(partSchema)) {
            if (shouldLog) {
                this.logInfo("soap:body part '" + partName + 
                             "' is complex type with schema: " + partSchema +
                             " has value: " +
                             (this.logIsDebugEnabled("xmlSerialize") ? 
                                    this.echo(data) : this.echoLeaf(data)), "xmlSerialize");
            }
            // a wsdl:part can specify complex type via @type or @element.  With @type, the
            // generated tag should match the part name.  With @element, it should match the
            // element name, which the partSchema will use if we just pass tagName as null.
            var tagName = partField.xsElementRef ? null : partName;
            return partSchema.xmlSerialize(data, flags, indent, tagName);
        } else {
            // if a simpleType part receives complex type data, assume they tried to specify
            // data as for example:
            //   { partName : value}
            // .. or ..
            //   { simpleTypeElementName : value }
            // .. and automatically find the simpleType value under those property names if
            // present
            if (data != null && !isc.DS.isSimpleTypeValue(data)) {
                 data = data[partField.name] || data[partName] || data;
            }

            if (shouldLog) {
                this.logInfo("soap:body part '" + partName + 
                             "' is of simple type '" + partField.type + "'" +
                             " has value: '" + this.echoLeaf(data) + "'", "xmlSerialize");
            }

            // if wsdl:part referred via element= to an element of simpleType in an imported
            // schema file.  This becomes a trivial isc.SimpleType but carries the correct
            // namespace.
            var simpleType = this.getType(partField.type),
                partNamespace = partField.partNamespace;
            if (!partNamespace && simpleType && simpleType.schemaNamespace) {
                partNamespace = simpleType.schemaNamespace;
            }

            //this.logWarn("partField: " + this.echo(partField) + 
            //             ", simpleType: " + this.echo(simpleType) + 
            //             ", partNamespace: " + partNamespace);

            return this._serializeSimpleTypeTag(partField.name || partName, partField, data,
                                                partNamespace, flags);
        }
    },


    // for a WSDLMessage, get the field that correspondes to a message <part>.
    // the field for a wsdl:part with @element will have field.name matching the
    // @element name (correct for serialization) but will also have a partName
    // attribute matching the "part" from the operation.inputHeaders/outputHeaders.
    // See schemaTranslator.xsl.
    getPartField : function (partName) {
        var partField = isc.getValues(this.getFields()).find("partName", partName);
        if (partField != null) return partField;
        return this.getField(partName);
    },
    getSoapEnd : function (dsRequest) {
        // allow centralized overrides of getSoapStart/End via overrides on a WebService object
        var service = this.getWebService(dsRequest);
        if (service.getSoapEnd) return service.getSoapEnd(dsRequest);

        return isc.DataSource._soapEnvelopeEnd;
    },
    
    getSoapStyle : function (dsRequest) {
        if (!this.hasWSDLService(dsRequest)) return "document";
        return this.getWebService(dsRequest).getSoapStyle(this.getWSOperation(dsRequest).name);
    },

    // get all the flags this DataSource should use for XML serialization, based on a
    // DSRequest and settings in the relevant operationBinding and the DataSource as a whole
    getSerializeFlags : function (dsRequest, flags) {
        flags = isc.addProperties({
            soapStyle : this.getSoapStyle(dsRequest)
        }, flags);

        // a few properties can be set on the operationBinding
        var operationBinding = this.getOperationBinding(dsRequest);
        
        // if useFlatFields is set, assume flat inbound data to be mapped automatically to
        // hierarchical XML message
        flags.flatData = this._firstNonNull(dsRequest.useFlatFields, 
                                            operationBinding.useFlatFields, 
                                            this.useFlatFields); 
        flags.recursiveFlatFields = this._firstNonNull(dsRequest.recursiveFlatFields,
                                                       operationBinding.recursiveFlatFields, 
                                                       this.recursiveFlatFields); 

        // ability to change the default textContentProperty for one serialization run
        flags.textContentProperty = this._firstNonNull(dsRequest.textContentProperty,
                                                       operationBinding.textContentProperty);

        // serialization of DS metadata: "startRowTag" et al can be set on the operationBinding schema,
        // and that tag will take on the value of dsRequest.startRow
        flags.dsRequest = dsRequest;
        flags.startRowTag = operationBinding.startRowTag || this.startRowTag;
        flags.endRowTag = operationBinding.endRowTag || this.endRowTag;
        return flags;
    },

    // Serializing Objects as XML
    // ---------------------------------------------------------------------------------------
    // Overall: we go through declared subelements, where the value of each subelement is the
    // value of the same named property in the object being serialized.  For complexType
    // subelements (described by another DataSource), we recurse.
    // - flatData mode: we assume the inbound structure is an object with properties that hold
    //   values for same-named subelements at various levels in the structure.  This is for
    //   flattening structures that have gratuitous nesting and are reasonably editable in a
    //   single form.
    // - ordering issues: if the XML output should be something like:
    //   <page>
    //      <section ../>
    //      <header ../>
    //      <section ../>
    //   .. where a subelement name appears *twice* or more, we have no object representation
    //   for this.  One possibility here is a marker on a field that indicates it serializes
    //   it's contents without outputting a tag for the fieldName itself (this would be the
    //   same or a related marker to what would cause immediate Canvas-typed subelements to be
    //   automatically binned in Canvas.children by default).

    // NOTE: there are a few reasons why schema is a necessity in XML serialization:
    // - SOAP
    //   - namespace requirements - eg must be able to tell whether an element definition was
    //     declared at top-level in the WSDL file, which means it requires namespacing
    //     (regardless of whether it appears in a given message)
    //   - "encoded" messages where xsi:type attributes are required pervasively - no way to
    //     detect the appropriate type from live JS data
    // - XML Schema (including it's use in SOAP)
    //   - order dependencies - XML schema basically forces you to declare an order even if you
    //     don't want one, and that order must be followed or the message is considered invalid
    // - ISC Widget XML
    //   - type:multiple fields and ValueMaps have custom encoding

    //> @object SerializationContext
    // Flags for XML serialization
    // @treeLocation Client Reference/Data Binding/DataSource
    // @visibility external
    //<
    
    //> @attr serializationContext.useFlatFields (boolean : null : IR)
    // Enables flat serialization mode, as described for +link{dsRequest.useFlatFields}.
    // @visibility external
    //<
    
    //> @attr serializationContext.xmlExplicitType (boolean : null : IR)
    // If specified, include explicit type information using <code>xsi:type</code>
    // attributes on the generated nodes for any attributes where no explicit field type is
    // specified in the dataSource, or +link{dataSourceField.serializeXMLType} is set to true.
    // The type written out will be determined based on the specified field type (if present)
    // otherwise on the type of the data being serialized.
    // <P>
    // Note that a field type of <code>"any"</code> is treated the same as a field with
    // no specified type - the <code>xsi:type</code> flag will be generated by default, and
    // will have a value derived from the type of the JavaScript value being serialized.
    //
    // @visibility internal
    //<

    //> @attr dataSourceField.serializeXMLType (boolean : null : IR)
    // Should an explicit "xsi:type" attribute be included in in the entry for this field
    // to indicate the data type when
    // +link{dataSource.xmlSerialize(),serializing data as XML}? Has no effect
    // unless +link{serializationContext.xmlExplicitType} is set to true.
    //
    // @visibility internal
    //<

    //> @method dataSource.xmlSerialize() [A]
    // Serialize a JavaScript object as XML.
    // <P>
    // The JavaScript Object passed to +link{method:DataSource.xmlSerialize()} becomes an XML
    // element named after the +link{attr:DataSource.tagName} (or +link{attr:DataSource.ID} if
    // tagName is unset).  Each property of the object becomes a subElement.  For example,
    // using a DataSource to serialize like this:
    // <pre>
    //     var inputObject = {
    //        startRow : 5,
    //        endRow : 50,
    //        data : [
    //           { field1 : "value1", field2: new Date() },
    //           { field1 : "value3", field2: null }
    //        ]
    //     };
    //     var myDS = isc.DataSource.create({ tagName:"DSRequest" });
    //     myDS.xmlSerialize(inputObject);
    // </pre>
    // .. produces the following XML:
    // <pre>
    //     &lt;DSRequest&gt;
    //         &lt;startRow&gt;5&lt;/startRow&gt;
    //         &lt;endRow&gt;50&lt;/endRow&gt;
    //         &lt;data&gt;
    //             &lt;field1&gt;value1&lt;/field1&gt;
    //             &lt;field2&gt;2005-10-14T18:01:16&lt;/field2&gt;
    //         &lt;/data&gt;
    //         &lt;data&gt;
    //             &lt;field1&gt;value3&lt;/field1&gt;
    //             &lt;field2&gt;&lt;/field2&gt;
    //         &lt;/data&gt;
    //     &lt;/DSRequest&gt;
    // </pre>
    // <P>
    // Various properties on the DataSource and DataSourceField can affect how serialization is
    // performed, see for example +link{DataSource.tagName},
    // +link{DataSource.schemaNamespace}, 
    // +link{DataSourceField.xmlAttribute}, 
    // +link{DataSourceField.multiple} and +link{DataSourceField.childTagName}.
    // By setting the +link{dataSourceField.type,type of a field} to the ID of another
    // DataSource which has further XML serialization settings, you can control serialization of
    // nested structures.
    // <P>
    // If you are working with a WSDL-described web service, XML serialization is performed
    // automatically by APIs like +link{method:WebService.callOperation()} - you only need to
    // know about serialization in order to understand how to put together JavaScript data that
    // will fill in an XML message properly, and for simple messages, setting
    // +link{dsRequest.useFlatFields} makes that unnecessary as well.   
    // <P>
    // <b>Note:</b> when trying to send data to a web service, it is best to avoid putting
    // together any XML yourself, instead modify the JavaScript data being fed to SmartClient's
    // SOAP engine.  This is because the WSDL and SOAP rules for correctly namespacing and
    // encoding Web Service messages are very complex and are subject to change with new
    // versions of the web service you are contacting, whereas the data itself is easy to
    // manipulate and less likely to change.
    // <P>
    // To troubleshoot message formation, you can set the log category "xmlSerialize" to 
    // <code>INFO</code> or <code>DEBUG</code> level in order to see diagnostics about XML
    // message formation, and you can use the RPC tab in the Developer Console to see the
    // actual messages being passed to web services.
    //
    // @param data (any) data to be serialized
    // @param flags (SerializationContext) options for the serialization engine
    // @return (String) data as serialized to XML
    //
    // @visibility xmlBinding
    //<
    xmlSerialize : function (data, flags, indent, tagName) {
        if (!flags) flags = {};

        if (flags.useFlatFields) flags.flatData = true;

        // qualifyAll, if set, should cascade to child elements which would not otherwise be
        // namespaced (because they were not declared as top level).  However when we are done
        // with this element, if we enabled qualifyAll we need to turn it back off
        var schemaSet = this.getSchemaSet(),
            enabledQualifyAll = (flags.qualifyAll == null);
        if (schemaSet && schemaSet.qualifyAll) {
            flags.qualifyAll = true;
        }

        
        var topLevel;
        if (flags.nsPrefixes == null) {
            var userPrefixes = this.xmlNamespaces ? isc.makeReverseMap(this.xmlNamespaces) : null;
            flags.nsPrefixes = isc.addProperties({}, userPrefixes);
            topLevel = true;
        }

        
        // enable xmlSchemaMode for any calls to xmlSerialize
        var oldXMLSchemaMode = isc.Comm.xmlSchemaMode;
        isc.Comm.xmlSchemaMode = true;
        // disable xsi:type unless it's been set explicitly
        
		
        
        var resetOmitXSI, omitXSIOldVal = null;
        if (flags.xmlExplicitType) {
        	resetOmitXSI = true;
        	omitXSIOldVal = isc.Comm.omitXSI;
			isc.Comm.omitXSI = false;
        } else {
        	if (isc.Comm.omitXSI == null) {
        		resetOmitXSI = isc.Comm.omitXSI = true;
        	}
		}

        var result = this._xmlSerialize(data, flags, indent, tagName, topLevel);

        // reset qualify all
        if (enabledQualifyAll) flags.qualifyAll = null;

        // restore global xml serialize settings
        isc.Comm.xmlSchemaMode = oldXMLSchemaMode;
		if (resetOmitXSI) isc.Comm.omitXSI = omitXSIOldVal

        return result;
    },

    _xmlSerialize : function (data, flags, indent, tagName, topLevel) {
        if (isc._traceMarkers) arguments.__this = this;
    
        if (this.logIsDebugEnabled("xmlSerialize")) {
            this.logDebug("schema: " + this + 
                         " serializing: " + this.echo(data) +
                         " with flags: " + this.echo(flags), "xmlSerialize");
        }

        var qualify = this.mustQualify || flags.qualifyAll,
            tagName = tagName || this.tagName || this.ID;

        
        if (!flags.ignoreConstructor) {
            var dataClassName;
            if (data != null && (data._constructor || isc.isAn.Instance(data))) {
                var dataClassName = isc.isAn.Instance(data) ? data.Class : data._constructor;
            }
        }

        // if we're passed an atomic value like a String, we definitely don't want to iterate
        // down a String's properties and pick up functions like "startsWith", the best we can
        // do is return the String inside our tags
        if (isc.DS.isSimpleTypeValue(data)) {

            if (isc.isA.String(data) && isc.startsWith(data,"ref:")) {
                return "<" + tagName + " ref=\"" + data.substring(4) + "\"/>";
            }

            this.logDebug("simple type value: " + this.echoLeaf(data) + 
                          " passed to xmlSerialize on " + this, "xmlSerialize");
            return isc.Comm._xmlSerialize(tagName ||  this.tagName || this.ID, data);
        }

        // if it's an instance, also trim it down to serializable fields
        
        if (isc.isAn.Instance(data)) data = data.getSerializeableFields();

        if (isc.isAn.Array(data) && !this.canBeArrayValued) return this.map("xmlSerialize", data, flags, indent).join("\n");

        var output = isc.SB.create(),
            indent = indent || "";

        output.append("\r", indent);

        var schemaNamespace;
        if (qualify) {
            //this.logWarn("schema: " + this + " parentNS: " + flags.parentSchemaNamespace);
            // XML Schema namespacing rules: if a subelement declares its structure via a "ref"
            // attribute pointing to another *Element* definition, the namespace of the
            // referenced element must be used.  If the subelement declares its structure via a
            // "type" attribute, the namespace of the parent element must be used.
            schemaNamespace = (this.isA("XSComplexType") ? flags.parentSchemaNamespace : null) 
                                    || this.schemaNamespace;
        }
		
		var xsiType;
		if (flags.xmlExplicitType) xsiType = null;
		else xsiType = this.ID;
		
        // leave the tag open..  
        output.append(
		    isc.Comm._xmlOpenTag(tagName, xsiType, schemaNamespace,
                                 flags.nsPrefixes, true)
        );

        // serialize attributes, if any.  Note only a field declared as an attribute
        // (field.xmlAttribute="true") ever becomes an attribute - undeclared data becomes
        // subelements
        data = this.serializeAttributes(data, output, flags);
        //this.logWarn("after serialize attributes, data is: " + this.echo(data));

        // if serializing an instance, save the actual class as "constructor" if it differs
        // from the schema we're using for serialization
        if (dataClassName && tagName != dataClassName) {
        	if (isc[dataClassName] != null) {
	            output.append(" constructor=\"", dataClassName, "\"");
	        
			} else {
				output.append(" _constructor=\"", dataClassName, "\"");
			}	            
        }

        var subElements;
        if (data != null) {
            subElements = this.xmlSerializeFields(data, flags, indent+"    ");
        }

        // if this is the top-level element, write out all the namespace prefixes encountered
        // during serialization
        if (topLevel) {
            output.append(this.outputNSPrefixes(flags.nsPrefixes, indent+"     "));
        }

        // NOTE: one time flag to prevent trailing ident, for use eg if there is text content
        var suppressIndent = this._suppressIndent;
        this._suppressIndent = null;

        // use a short tag if possible
        if (subElements == null || isc.isAn.emptyString(subElements)) {
            output.append("/>");
            return output.toString();
        }

        output.append(">", subElements, 
                      // NOTE: one time flag to prevent if there is text content, don't add whitespace/newline inside tag
                      (suppressIndent ? "" : "\r" + indent));
        
        output.append(isc.Comm._xmlCloseTag(tagName, schemaNamespace, flags.nsPrefixes));

        return output.toString();
    },

    outputNSPrefixes : function (prefixes, indent) {
        delete prefixes._nsCount;
        prefixes = isc.makeReverseMap(prefixes);
        var nsDefs = isc.xml._generateNamespaces(prefixes, null, indent+"        ");
        return nsDefs;
    },

    serializeAttributes : function (data, output, flags) {
        var fieldNames = this.getFieldNames(),
            originalData = true;
        
        for (var i = 0; i < fieldNames.length; i++) {
            var fieldName = fieldNames[i],
                field = this.getField(fieldName);
            if (field.xmlAttribute && ((data && data[fieldName] != null) || field.xmlRequired)) {
                // if we run into attributes, shallow copy the data so we can remove already
                // serialized fields without affecting the data the developer passed to
                // xmlSerialize()
                if (originalData) {
                    data = isc.addProperties({}, data);
                    originalData = false;
                }

                var value = data[fieldName];
                if (flags && flags.spoofData) value = this.getSpoofedData(field);

                output.append(" ", fieldName, "=\"", 
                        this._serializeSimpleTypeValue(field, value), 
                        "\"");
                delete data[fieldName]
            }
        }
        return data; // return the (possibly copied) data
    },

    xmlSerializeFields : function (data, flags, indent) {
        var output = isc.SB.create(),
            flags = flags || isc.emptyObject,
            flatData = flags.flatData,
            spoofData = flags.spoofData,
            indent = indent || "";

        // output each known field *in order*
        var data = isc.addProperties({}, data); // copy data so we can delete known fields

        
        if (data.__ref) {
            delete data.__ref;
            delete data.__module;
        }

        var fields = this.getFields();
        for (var fieldName in fields) {
            var field = this.getField(fieldName),
                value = data[fieldName],
                    
                fieldIsComplex = this.fieldIsComplexType(fieldName);

            var childData = data[fieldName];
            if (flags.startRowTag == field.name && childData == null) {
                childData = flags.dsRequest ? flags.dsRequest.startRow : null;
            } else if (flags.endRowTag == field.name && childData == null) {
                childData = flags.dsRequest ? flags.dsRequest.endRow : null;

            // flatData mode: we assume the inbound data is an object with properties
            // that hold values for same-named subelements at various levels in the
            // structure.  This is for flattening structures that have gratuitous nesting
            // and are reasonably editable in a single form.
            // Note that even with flatData=true, "data" may have nested objects, intended
            // as data for a same-named complexType subelement, so flatData amounts to always
            // passing data to child elements of complexType
            } else if (fieldIsComplex && flatData && childData == null) {
                childData = data;
            }
    
            //if (this.logIsDebugEnabled("xmlSerialize")) {
            //    this.logDebug("for field: '" + fieldName + 
            //                 "' serializing data: " + this.echo(childData), "xmlSerialize");
            //}

            // NOTE: the xml notion of required is just that an element needs to be there - not
            // that it has to be non-empty
            var mustOutput = (field.xmlRequired && !field.xmlAttribute) || 
                                (data[fieldName] != null || (spoofData && !field.xmlAttribute));

            if (flatData && fieldIsComplex) {
                // HACK: for flatData, if a tag is not marked required, we can't easily
                // tell whether there will be output since subobjects may produce output if
                // the data happens to intersect with some simpleType field at any level of
                // nesting, so we just try serializing and throw the result away if there's
                // nothing
                var tagSchema = this.getSchema(field.type),
                    // HACK: this serialization run doesn't produce output, so disable caching
                    // of NS prefixes
                    origPrefixes = isc.clone(flags.nsPrefixes),
                    fieldsOutput = tagSchema.xmlSerializeFields(childData, flags);
                //this.logWarn("fieldName: '" + fieldName + 
                //             "', mustOutput before flatData check: " + mustOutput + 
                //             ", fieldsOutput: " + this.echoLeaf(fieldsOutput));
                if (fieldsOutput != null && !isc.isAn.emptyString(fieldsOutput)) {
                    mustOutput = true;
                }
                flags.nsPrefixes = origPrefixes;
            }

            if (mustOutput) {
                
                if (flatData && fieldIsComplex && 
                    // non-simpleType data is provided
                    data[fieldName] != null && !isc.DS.isSimpleTypeValue(data[fieldName]) &&
                    !flags.recursiveFlatFields) 
                {
                    // NOTE: we intentionally addProperties() rather than clone() here, because
                    // we do want flags.nsPrefixes to have any additional nsPrefixes added to
                    // it
                    flags = isc.addProperties({}, flags);
                    flags.flatData = false;
                }
                output.append(this.xmlSerializeField(fieldName, childData, flags, indent));
            }
            // in flatData mode, we don't delete fields that we've output, because we don't
            // write out undeclared data at the end
            if (!flatData && data[fieldName] != null) delete data[fieldName];
        }

        // if there's any data left, tack them on the end, but *not* if this DataSource came
        // form XML Schema, in which case the extra data is sure to be invalid
        if (!flatData && !isc.isA.Schema(this)) {
            for (var fieldName in data) {
                output.append(this.xmlSerializeField(fieldName, data[fieldName], flags, indent));
            }
        }

        return output.toString();
    },

    _$Action:"Action",
    xmlSerializeField : function (fieldName, value, flags, indent) {
        var output = isc.SB.create(),
            field = this.getField(fieldName);
        // skip undeclared, generated properties
        if (field == null && (fieldName.startsWith("_") || fieldName.startsWith("$"))) return;

        var fieldType = (field ? field.type : null),
            flatData = flags && flags.flatData,
            spoofData = flags && flags.spoofData,
            indent = indent || "";

        if (spoofData) value = this.getSpoofedData(field);
    
        if (this.logIsDebugEnabled("xmlSerialize")) {
            this.logDebug("serializing fieldName: " + fieldName + 
                          " with type: " + fieldType + 
                          " with value: " + this.echo(value), "xmlSerialize");
        }

        // namespace the element if either everything must be qualifed or this particular field
        // is declared and must be qualified
        var namespace = ((field && field.mustQualify) || flags.qualifyAll ? 
                         this.getSchemaSet().schemaNamespace : null);

        // if this field matches the textContentProperty, and either a textContent field has
        // been declared or there are no elements, return the value of the textContentProperty
        // with no surrounding tag and no indent.
        // NOTE: this handling appears here so that it works with flatData and spoofData 
        var textContentProperty = flags.textContentProperty || this.textContentProperty,
            textContentField = this.getTextContentField();

        if (fieldName == textContentProperty && 
            (textContentField != null || !this.hasXMLElementFields(textContentProperty))) 
        {
            // one time flag prevents normal whitespace/indent from being added
            this._suppressIndent = true;
            return this._serializeSimpleTypeValue(textContentField, value);
        }

        // special case for "Action"s - get the iscAction we stored on the function object
        if (fieldType == this._$Action && value != null) {
            if (value.iscAction) {
                value = value.iscAction
            // sanity check for StringMethods (may not be necessary since addMethods() / 
            // addProperties() should have converted StringMethods to functions with iscAction 
            // stored on them
            } else if (isc.isA.StringMethod(value)) {
                value = value.value;                
            }
        }
		
		var xsiType = !flags.xmlExplicitType || this.shouldWriteSchemaType(field) ?
					this._getXMLSchemaType(field ? field.type : null, value) : null;

        var fieldStart = isc.Comm._xmlOpenTag(fieldName, xsiType, 
                                              namespace, flags.nsPrefixes),
            fieldEnd = isc.Comm._xmlCloseTag(fieldName, namespace, flags.nsPrefixes);

        //this.logWarn("schemaSet is: " + this.echo(schemaSet) + ", fieldStart: " + fieldStart);

        var values = isc.isAn.Array(value) ? value : [value];
        
        // If this DataSource has no field of the given name, see if we were passed another
        // DataSource to use as schema by the calling code.  RestDataSource uses this pattern
        // to allow it to control the overall style of XML serialization (the tag names to use
        // for criteria, criterion, etc) and the detailed serialization of individual fields 
        // (most critically for fields of type "date", "time" and "datetime")
        if (!field && flags.schema && flags.schema.getField) {
            field = flags.schema.getField(fieldName);
        }

        // for a DataSource type, ask the DataSource to serialize the value
        if (this.fieldIsComplexType(fieldName)) {
            var oldPSN = flags.parentSchemaNamespace;
            flags.parentSchemaNamespace = this.schemaNamespace;
            //if (field && field.xsElementRef) this.logWarn("looking up *element* only: " + fieldType);
    
            var ds = this.getFieldDataSource(field, field && field.xsElementRef ? "element" : null);
            //this.logWarn("complexType field: " + fieldName + 
            //             " with schema: " + ds +
            //             " has value: " + this.echo(value));

            if (field.multiple) {
                // if this is a field declaration with multiple=true, we write out a
                // container tag containing one or more tags for the values, where the values
                // have the typeName, eg
                //      <children> // container tag: fieldName
                //          <Canvas> // value tag: field dataSource
                //          </Canvas>
                //          <Canvas> 
                //          </Canvas>
                //      </children>
                output.append("\r", indent, fieldStart);
                for (var i = 0; i < values.length; i++) {
                                                  
                    output.append(ds.xmlSerialize(values[i], flags, indent+"    ", 
                                                  // allow the field to specify a name for 
                                                  // child tags, eg, ds.fields -> "field"
                    							  values[i] == null
                    							   ? field.childTagName 
                    							   : values[i]._tagName || field.childTagName));
                }
                output.append("\r", indent, fieldEnd);

            } else if (ds.canBeArrayValued && isc.isAn.Array(value)) {
                // very special case for the valueMap type, which when it appears as a simple
                // JS Array, shouldn't be considered multiple valueMaps.

                output.append(ds.xmlSerialize(value, flags, indent, fieldName));
            } else {
                // if this is a field that isn't declared multiple but has multiple values,
                // we write out one or more tags named after the field name since that's the
                // inbound format that would produce an array in a field not declared
                // multiple
                for (var i = 0; i < values.length; i++) {
                    var value = values[i];
                    if (value == null) { // null = empty tag
                        output.append("\r", indent)
                        output.append(fieldStart, fieldEnd);
                    // handle being unexpectedly passed a simple type.  If we're serializing a
                    // SOAP message, the resulting XML isn't going to adhere to the schema, but
                    // there's nothing we can do.
                    } else if (isc.DS.isSimpleTypeValue(value)) {
                        if (isc.isA.String(value) && isc.startsWith(value,"ref:")) {
                            output.append("\r", indent)
                            output.append(fieldStart);
                            var refTagName = (field ? field.childTagName || field.type : "value");
                            output.append("<", refTagName, " ref=\"", value.substring(4), "\"/>");
                            output.append(fieldEnd);
                        } else {
                            this.logWarn("simple type value " + this.echoLeaf(value) + 
                                         " passed to complex field '" + field.name + "'",
                                         "xmlSerialize");
                            output.append("\r", indent)
                            output.append(isc.Comm.xmlSerialize(fieldName, value));
                        }
                    } else {
                        output.append(ds.xmlSerialize(value, flags, indent, fieldName));
                    }
                }
            }
            flags.parentSchemaNamespace = oldPSN;

        // declared field of simple type
        } else if (field != null) {
            
            // if this field came from a simple type element, we have to use the namespace that
            // the element was defined in.  Note that both xs:simpleTypes and xs:elements of
            // simpleType become SimpleTypes in SmartClient - both essentially represent a
            // reusable atomic type.
            if (field.xsElementRef) {
                var simpleType = this.getType(field.type);
                //this.logWarn("in DataSource: " + this + 
                //             ", simpleType field: " + this.echo(field) + 
                //             ", got simpleType: " + this.echo(simpleType));
                if (simpleType && simpleType.schemaNamespace)
                {
                    namespace = simpleType.schemaNamespace;
                }
            }
    
            if (field.multiple) {
                output.append("\r", indent, fieldStart, "\r");
                for (var i = 0; i < values.length; i++) {
                    // field.childTagName dictates the childTagName.  If it's null a generic
                    // name will be used
                    output.append(
                         this._serializeSimpleTypeTag(field.childTagName, field,
                                                      values[i], namespace, flags),
                         "\r", indent);
                }
                output.append("\r", indent, fieldEnd, "\r");
            } else {
                for (var i = 0; i < values.length; i++) {
                    output.append(
                        "\r", indent, 
                        this._serializeSimpleTypeTag(fieldName, field,
                                                       values[i], namespace, flags));
                }
            }

        // undeclared field, simple or complexType
        } else {
            //this.logWarn("serializing values for undeclared field: " + fieldName + 
            //             ", values: " + this.echoAll(values));
            for (var i = 0; i < values.length; i++) {
                if (values[i] == null || isc.isAn.emptyObject(values[i])) {
                    output.append("\r", indent, fieldStart, fieldEnd);
                } else {
                    output.append("\r", indent, 
                                 isc.Comm._xmlSerialize(fieldName, values[i], indent,
                                                        { isRoot:false }));
                }
            }
        }

        return output.toString();
    },
    
    // When performing XML Serialization, should we include an "xsi:type" tag indicating
    // the type of data for some field.
    // This method is only called in the case where xmlSerialize was passed a context with
    // the "xmlExplicitType" flag set, and returns true for fields where we do want
    // an "xsi:type" entry - which is basically fields of variable type like the
    // 'any' field in a criterion object.
    shouldWriteSchemaType : function (field) {
    	// No field - we don't have schema type info, so write out an explicit type based
    	// on the value, so the parser will know how to regen JS on a round-trip.
    	if (!field) return true;
    	// Respect explicit "serializeXMLType" if set
    	if (field.serializeXMLType != null) return field.serializeXMLType;
    	else {
    		// field has no specified type on the schema, so pick it up from the value.
    		return (field.type == null || field.type == "any");
    	}
    },

    
    _serializeSimpleTypeTag : function (fieldName, field, value, namespace, flags) {
        // NOTE: fieldName is passed as "null" for multiple:true fields, indicating use the
        // generic "value" for simple values, and let Comm.xmlSerialize pick a name if an
        // Object has been (erroneously) passed to a simple type field.  Comm.xmlSerialize will
        // use the Object's type as a tag name, defaulting to the generic "Object"

        var type = field.type,
            flags = flags || {};

        
        if (isc.isAn.Object(value) && !isc.isA.Function(value._xmlSerialize)) {
            return isc.Comm.xmlSerialize(fieldName || null, value);
        } else {
            
            
            var type = !flags.xmlExplicitType || this.shouldWriteSchemaType(field) ?
            			this._getXMLSchemaType(type, value) : null;

            // if the value is null and the field is nillable, send an empty element
            // (<element/>) with xsi:nil=true rather than an element with empty content
            // (<element></element>).  Note that this distinction is key for eg boolean or
            // other restricted types, where empty content is actually a bad value.
            if (value == null && field.nillable) {
                var tagName = fieldName || "value";
                return isc.Comm._xmlOpenTag(tagName, null, namespace, flags.nsPrefixes, true)
                        + " xsi:nil=\"true\"/>";
            }

            if (isc.isA.Date(value)) {
                // per XML Schema spec, 
                // - type == "date" omits the time 
                // - type == "time" omits the date and shows just the time
                // - type == "datetime" shows both.
                value = value.toSchemaDate(field.type);
            } else if (value != null && value._xmlSerialize) {
                return value._xmlSerialize(fieldName, type, namespace); 
            } else {
                // makeXMLSafe covers String, null (as empty string), Number and Boolean
                value = isc.makeXMLSafe(value);
            }
            return isc.Comm._xmlValue(fieldName || "value", value, type, namespace,
                                      flags.nsPrefixes);
        }
    },

    // serialize a simple type value without surrounding tags, for use in attributes and for
    // textContent fields.
    // If value is not actually of simple type, toString() is called on it (best we can do in a
    // context where we are not allowed to generate elements).
    // Pseudo-bug: the internal _xmlSerialize override is factored incorrectly because it
    // always returns a tag, so it cannot be called for an attribute value.
    _serializeSimpleTypeValue : function (field, value) {
        if (isc.isA.Date(value)) {
            return value.toSchemaDate(field ? field.type : null);
        } else {
            return isc.makeXMLSafe(value);
        }
    },

    // translate dataSource field types to their XML Schema equivalents, where we have synonyms
    _getXMLSchemaType : function (type, value) {
    	// if a field has no type specified and we were passed a value, derive from
    	// the value.
    	
    	if (value != null && (type == null || type == "any")) {
    		if (value == null) return null;
    		if (isc.isA.Date(value)) {
    			return value.logicalDate ? "date" 
    			                         : (value.logicalTime ? "time" : "datetime");
    		} else if (isc.isA.Number(value)) {
    			if (parseInt(value) == value) {
    				return "float";
    			} else {
    				return "integer";
    			}
    		} else if (isc.isA.Boolean(value)) {
    			return "boolean";
    		} else if (isc.isA.String(value)) {
    			return "text";
    		}
    	}
        switch (type) {
            case "integer":
                return "int";
            case "number" :
                return "long";
            default:
                return type;
        }
    },

    xmlSerializeSample : function () {
        return this.xmlSerialize({}, { spoofData:true });
    },

    getSpoofedData : function (field) {
        if (!field) return "textValue";

        // return an object for object-type fields (to avoid following pathway of simpleType
        // passed to complexType field)
        if (this.getSchema(field.type) != null) return {};

        // return arrays for multiple fields 
        if (field.multiple) {
            field = { type : field.type };
            return [this.getSpoofedData(field), this.getSpoofedData(field)];
        }

        // for valueMapped fields, return a value at random
        if (field.valueMap) {
            var values = !isc.isAn.Array(field.valueMap) ? 
                    isc.getKeys(field.valueMap) : field.valueMap;
            return values[Math.round(Math.random() * (values.length-1))];
        }

        // return various simpleType values
        var baseType = isc.SimpleType.getBaseType(field.type);
        switch (baseType) {
            case "boolean" :
                return (Math.random() > 0.5);
            case "integer" :
            case "int" :
            case "number" :
                var min = 0, max = 10;
                if (field.validators) {
                	// check for validator._constructor as well as validator.type. This handles
                	// validators converted from Validator sub-object schema
                    var rangeValidator = field.validators.find("type", "integerRange") ||
                    					 field.validators.find("_constructor", "integerRange");
                    if (rangeValidator) {
                        this.logWarn(field.name + " has " + 
                                     rangeValidator.type + " validator " + 
                                     " with min " + rangeValidator.min + 
                                     " and max " + rangeValidator.max);
                        min = rangeValidator.min || 0;
                        // NOTE: this means if max is < 0 we always pick it exactly
                        max = rangeValidator.max || Math.min(min, 10);
                        if (min > max) min = max;
                    }
                }
                return Math.round(min + (Math.random() * (max-min)));
            case "float" :
            case "decimal" :
            case "double" :
                var min = 0, max = 10, precision = 2;
                if (field.validators) {
                    var rangeValidator = field.validators.find("type", "floatRange") ||
                    					 field.validators.find("_constructor", "floatRange");
                    if (rangeValidator) {
                        this.logWarn(field.name + " has " + 
                                     rangeValidator.type + " validator " + 
                                     " with min " + rangeValidator.min + 
                                     " and max " + rangeValidator.max);
                        min = rangeValidator.min || 0;
                        // NOTE: this means if max is < 0 we always pick it exactly
                        max = rangeValidator.max || Math.min(min, 10);
                        if (min > max) min = max;
                    }
                    var precisionValidator = field.validators.find("type", "floatPrecision") ||
                    						 field.validators.find("_constructor", "floatPrecision");
                    if (precisionValidator) {
                        precision = precisionValidator.precision || 2;
                    }
                }
                return (min + (Math.random() * (max-min))).toFixed(precision);
            case "date" :
            case "time" :
            case "datetime" :
                var date = new Date();
                if (field.validators) {
                    var rangeValidator = field.validators.find("type", "dateRange") ||
                    					 field.validators.find("_constructor", "dateRange");
                    if (rangeValidator) {
                        this.logWarn(field.name + " has " + 
                                     rangeValidator.type + " validator " + 
                                     " with min " + rangeValidator.min + 
                                     " and max " + rangeValidator.max);
                        // NOTE: picking a random date between min and max would require at
                        // least zeroing out either time or date values for pure date or pure
                        // time value fields
                        if (rangeValidator.min) date = rangeValidator.min;
                        else if (rangeValidator.max) date = rangeValidator.max;
                    }
                }
                return date;
            default:
                
                return "textValue";
        }
    },


    //>EditMode
    // extract saveable data from a live DataSource
	getSerializeableFields : function (removeFields, keepFields) {
	    var props = this.Super("getSerializeableFields", arguments);

        var fields = props.fields;
        // switch back from Map to Array
        fields = isc.getValues(fields);
        // remove type-based validators automatically inserted into validators Array
        for (var i = 0; i < fields.length; i++) {
            var field = fields[i] = isc.addProperties({}, fields[i]);
            var validators = field.validators;
            if (validators) {
                field.validators = validators.findAll("_generated", null);
                if (field.validators == null) delete field.validators;
            }
        }
        props.fields = fields;
        
        return props;
    },
    //<EditMode

    // XML Response handling
    // ---------------------------------------------------------------------------------------

    // take the XML response from a web service and ask the DataSource to provide an Array of
    // XMLElements from it
    _handleXMLReply : function (xmlDoc, xmlText, rpcResponse, rpcRequest) {
        var xmlData = xmlDoc,
            dsRequest = rpcRequest._dsRequest,
            operationBinding = this.getOperationBinding(dsRequest),
            opNamespaces;
            
        // Don't try to handle the response unless it's valid
        if (rpcResponse.status < 0) {
            var data = xmlText || rpcResponse.data;
            this._completeResponseProcessing(data, {status: rpcResponse.status, data:data},
                                             dsRequest, 
                                             rpcResponse, rpcRequest);
            return;
        }
        
        // add namespaces to the xmlDoc to make subsequent selection easier
        if (xmlData) {
            if (operationBinding.wsOperation) {
                var service = this.getWebService(dsRequest),
                    opNamespaces = service.getOutputNamespaces(operationBinding.wsOperation);
                xmlData.addNamespaces(opNamespaces);
            }
            xmlData.addNamespaces(this.xmlNamespaces);
            xmlData.addNamespaces(operationBinding.xmlNamespaces);
        }

        // Ensure the full set of defined namespaces is available to all xPaths
        // executed during dsResponseFromXML.  Currently we associate all namespaces with the
        // XMLDoc, which means they'll be available for selectNodes() called directly with the
        // xmlDoc, but not for selectNodes() relative to an element (eg valueXPath).
        // To fix this pervasively such that any call to selectNodes() with an element works,
        // we should either tag or wrap all native XMLElements returned from selectNodes() so
        // that we always find the right set of namespaces.
        var xmlNamespaces = isc.addProperties({}, opNamespaces, 
                                               this.xmlNamespaces,
                                               operationBinding.xmlNamespaces);
                                          
        // NOTE: it's key to pass the dsRequest to dsResponseToXML since among other things
        // custom message formats may have the server just return a range of rows without
        // reminding the client what the range is
        // dsResponseFromXML can be asynchronous (depending on resultBatchSize)
        this.dsResponseFromXML(xmlData, dsRequest, xmlNamespaces, 
                                {target:this, methodName:"_completeHandleXMLReply",
                                    xmlData:xmlData, dsRequest:dsRequest,
                                    rpcRequest:rpcRequest, rpcResponse:rpcResponse
                                }
        ); 
        
    },
    
    _completeHandleXMLReply : function (dsResponse, callback) {
        this._completeResponseProcessing(callback.xmlData, dsResponse, callback.dsRequest, 
                                                  callback.rpcResponse, callback.rpcRequest);
    },

    //> @method DataSource.dsResponseFromXML() [A]
    // Generate a DSResponse from an XML Document already present on the client.
    // Makes use of <code>recordXPath</code> specified on the DataSource or the appropriate
    // operationBinding to select the appropriate set of nodes, and extracts record data 
    // as described in +link{DataSource.recordsFromXML()}. Extracted data is then assembled
    // into a <code>DSResponse</code> object, available on the callback.
    // @param xmlData (XML Document) XML Data from with the DSResponse will be generated
    // @param dsRequest (DSRequest) Request object
    // @param xmlNamespaces (object) Namespace prefixes to apply when extracting data
    // @param callback (Callback) callback to fire when the response has been generated. Takes
    //  a single parameter <code>dsResponse</code> - the generated response object.
    // 
    // @visibility internal
    //<
    
    dsResponseFromXML : function (xmlData, dsRequest, xmlNamespaces, callback) {
        if (xmlData) {
            this.selectRecords(
                xmlData, dsRequest, 
                {target:this, methodName:"_completeDSResponseFromXML", dsRequest:dsRequest,
                 callback:callback, xmlData:xmlData, xmlNamespaces:xmlNamespaces}
            );
        } else {
            this._completeDSResponseFromXML([], dsRequest, xmlNamespaces, callback);
        }
    },
    
    _completeDSResponseFromXML : function (data, dsRequest, xmlNamespaces, callback) {
        // handle the case where this is fired asynchronously as a callback - 
        // pick up the dsRequest from the 2nd arguments (the callback object)
        if (!callback && dsRequest.callback) callback = dsRequest.callback;
        if (dsRequest.xmlNamespaces) xmlNamespaces = dsRequest.xmlNamespaces;
        if (dsRequest.dsRequest) dsRequest = dsRequest.dsRequest;
        if (xmlNamespaces == null) xmlNamespaces = this.xmlNamespaces;
        // default startRow as matching what was requested
        var dsResponse = { 
            data : data,
            startRow: dsRequest.startRow || 0,
            status : 0
        };
        dsResponse.endRow = dsResponse.startRow + Math.max(0, data.length);
        // assume dataset is complete by default
        dsResponse.totalRows = Math.max(dsResponse.endRow, data.length);
        var xmlData = callback.xmlData;
           
        // added null check on xmlData for the empty response case
        if (xmlData) {
            if (this.totalRowsXPath) {
                dsResponse.totalRows = 
                    isc.xml.selectNumber(xmlData, this.totalRowsXPath, xmlNamespaces, true);
            }
            if (this.startRowXPath) {
                dsResponse.startRow = 
                    isc.xml.selectNumber(xmlData, this.startRowXPath, xmlNamespaces, true);
                dsResponse.endRow = dsResponse.startRow + Math.max(0, data.length);
            }
            if (this.endRowXPath) {
                dsResponse.endRow = 
                    isc.xml.selectNumber(xmlData, this.endRowXPath, xmlNamespaces, true);
                // if there's no XPath specified for the startRow, derive it from endRow - number
                // of records returned
                if (!this.startRowXPath) {
                    dsResponse.startRow = dsResponse.endRow - Math.max(0, data.length);
                }
            }
    
            // an XPath is unlikely to be able to really translate 3rd party error codes to ISC
            // error codes, but might allow a first cut of error vs no error
            if (this.statusXPath) {
                dsResponse.status = parseInt(
                    isc.xml.selectScalar(xmlData, this.statusXPath, xmlNamespaces, true));
            }
            if (this.errorSchema) {
                dsResponse.errors = this.errorSchema.selectRecords(xmlData, dsRequest);
            }
        }
        if (callback) this.fireCallback(callback, "dsResponse", [dsResponse, callback])
        return dsResponse;
    },

    
    
    //>@attr dataSource.resultBatchSize (int : 150 : IRWA)
    // Very advanced: for servers that do not support paging, and must return large numbers of XML
    // records in one HTTP response, SmartClient breaks up the processing of the response
    // in order to avoid the "script running slowly" dialog appearing for an end user.
    // <P>
    // If you have a relatively small number of records with a great deal of properties or
    // subobjects on each record, and you have not set +link{dropExtraFields} to eliminate unused
    // data, and you see the "script running slowly" dialog, you may need to set this number
    // lower.
    // @serverDS allowed
    // @visibility external
    //<
    
    resultBatchSize:150,
    
    // extract records from an XML document using the recordXPath, and transform to JS using
    // the schema
    // @visibility xmlBinding
     
    selectRecords : function (xmlData, dsRequest, callback) {
        var elements = this.selectRecordElements(xmlData, dsRequest);
        var operationBinding = this.getOperationBinding(dsRequest),
            schema = this.getSchema(operationBinding.responseDataSchema) || this;
        return schema.recordsFromXML(elements, callback);
    },
    
    //> @method dataSource.recordsFromXML() [A]
    // Transform a list of XML elements to DataSource records.
    // <P>
    // <code>recordsFromXML()</code> will create a List of DataSource records in the form of
    // JavaScript objects.  The value for each field is extracted from the XML according to the
    // rules described under +link{attr:DataSourceField.valueXPath}. 
    // <P>
    // Derived JavaScript values for each field will be the appropriate JavaScript type, eg,
    // for a field of "date" type, the JS value will be a JavaScript Date object.
    // <P>
    // Note that if the set of nodes exceeds +link{DataSource.resultBatchSize}, this method
    // will break the XML processing logic into multiple threads and therefore complete
    // asynchronously. In this case the return value will be null, and the callback parameter
    // must be used to process the extracted records.
    // 
    // @param elements (List of XMLElement) XML elements to transform, eg, the result of a
    //                                       call to +link{XMLTools.selectNodes()}
    // @param callback (Callback) Callback to fire when the transform completes. Takes a single
    //    parameter <code>records</code> - the array of DataSource records derived from the
    //    XML elements.
    //
    // @return (List) List of DataSource records derived from the XML elements, or null
    //   if this method completed the XML processing asynchronously.
    // @visibility xmlBinding
    //<
    recordsFromXML : function (elements, callback) {
        // handle being passed a single element, or a native nodeList (as returned by natively
        // calling selectNodes)
        // check elements and skip if null (empty response case)
        if (elements && !isc.isAn.Array(elements)) {
            // XML elements have no length attribute, but NodeLists do
            if (elements.length != null) elements = isc.xml._nodeListToArray(elements);
            // assume a single XML element
            else elements = [elements];
        }

        if (elements && this.transformResponseToJS) {
            if (elements.length > this.resultBatchSize) {                
                var fromXMLContext = {startingRow:0, callback:callback, elements:elements};
                return this._asyncRecordsFromXML(fromXMLContext);
                
            } 
            
            var attrMask = this.dropExtraFields ? this.getFieldNames() : null;
            
            elements = isc.xml.toJS(elements, attrMask, this);
            if (this.logIsDebugEnabled("xmlBinding")) {
                this.logDebug("transformed response: " + this.echoFull(elements) +
                              "xmlBinding");
            }
        }         
        
        if (callback) {
            this.fireCallback(callback, "records", [elements, callback]);
        }
        return elements;
    },
    
    _asyncRecordsFromXML : function (context) {
        
        var elements = context.elements, 
            startingRow = context.startingRow,
            callback = context.callback,
            endRow = Math.min(elements.length, startingRow + this.resultBatchSize),
            attrMask = this.dropExtraFields ? this.getFieldNames() : null;

        if (!context._recordsFromXML) {
            context._recordsFromXML = isc.xml.toJS(
                                        elements.slice(startingRow, endRow+1), attrMask, this);
        } else {
            var records = isc.xml.toJS(elements.slice(startingRow, endRow+1), attrMask, this);
            context._recordsFromXML.addList(records);
        }
        if (endRow < elements.length) {
            context.startingRow = endRow + 1;
            this.delayCall("_asyncRecordsFromXML", [context]);
        } else if (callback) {
            this.fireCallback(callback, "records", [context._recordsFromXML, callback]);
        }
    },

    // given xmlData return by a web service, return an Array of XML elements to be used as a
    // recordSet
    selectRecordElements : function (xmlData, dsRequest) {
        if (isc.isA.String(xmlData)) xmlData = isc.xml.parseXML(xmlData);
        

        // get the XPath or tagName to use to select record elements from the operationBinding
        var operationBinding = this.getOperationBinding(dsRequest);

        // precedence order for selecting records: opBinding.recordXPath, opBinding.recordName, 
        //                                         dataSource.recordXPath, dataSource.recordName
        // Slightly tricky because operationBinding defaults to the DataSource itself when
        // there is no operationBinding for the operation.
        var opXPath = operationBinding == this ? null : operationBinding.recordXPath, 
            opRecordName = operationBinding == this ? null : operationBinding.recordName,
            dsXPath = this.recordXPath,
            dsRecordName = this.recordName;

        // if we have a recordName and a WSDL service definition, and there is no overriding
        // recordXPath definition, use the WebService to select the records by type.
        if (opXPath == null && 
            (opRecordName != null || (dsXPath == null && dsRecordName != null)) && 
            this.hasWSDLService(dsRequest)) 
        {
            // if we have a WebService, allow the recordName to be used as a way of
            // auto-generating the recordXPath
            var service = this.getWebService(dsRequest);

            return service.selectByType(xmlData, 
                                        operationBinding.wsOperation || this.wsOperation,
                                        opRecordName || dsRecordName);
        }

        // use a recordXPath if specified, otherwise use recordName to do just a simple
        // getElementsByTagName
        var recordXPath = opXPath || dsXPath,
            elements;
        if (recordXPath) {
            elements = isc.xml.selectNodes(xmlData, recordXPath, this.xmlNamespaces);
            this.logDebug("applying XPath: " + recordXPath + 
                          (this.xmlNamespaces ? 
                           " with namespaces: " + this.echo(this.xmlNamespaces) : "") +
                          " got " + (elements ? elements.length : null) + " elements", "xmlBinding");
        } else {
            elements = [];
            var recordName = opRecordName || dsRecordName || this.ID;
            // NOTE: a tagName of "record" matches <foo:record> in Moz, but NOT in IE
            var nodeList = xmlData.getElementsByTagName(recordName);
            for (var i = 0; i < nodeList.length; i++) elements.add(nodeList[i]);
            this.logDebug("getting elements of tag name: " + recordName +
                          " got " + elements.length + " elements", "xmlBinding");
        }

        return elements;
    },

    // Common XML / JSON Response handling
    // ---------------------------------------------------------------------------------------
    
    _completeResponseProcessing : function (data, dsResponse, dsRequest,
                                                  rpcResponse, rpcRequest) 
    {
        // empty response case, set up dsResponse manually
       if (!dsResponse) {
           
           // set the status and httpResponseCode, or the saving of local data will fail
            dsResponse = {
                status: rpcResponse.status, 
                httpResponseCode: rpcResponse.httpResponseCode
            };
        }
        // note for "clientCustom" operations this method is fired directly from
        // processResponse() and rpcResponse / rpcRequest will be null
        if (rpcResponse != null && rpcRequest != null) {
            // make the httpResponseCode available on dsResponse - needed for REST implementations
            dsResponse.httpResponseCode = rpcResponse.httpResponseCode;
            dsResponse.httpResponseText = rpcResponse.httpResponseText;
            // grab the transactionNum - needed for relogin via resubmitTransaction()
            dsResponse.transactionNum = rpcResponse.transactionNum;
            // place the clientContext on the response
            dsResponse.clientContext = rpcRequest.clientContext;
            dsResponse.internalClientContext = rpcRequest.internalClientContext;
            dsResponse.httpHeaders = rpcResponse.httpHeaders;
        } else {
            dsResponse.clientContext = dsRequest.clientContext;
            dsResponse.internalClientContext = dsRequest.internalClientContext;
        }
        

        if (this.logIsInfoEnabled("xmlBinding")) {
            this.logInfo("dsResponse is: " + this.echo(dsResponse), "xmlBinding");
        }

        
        dsResponse.context = rpcRequest; 

        // allow customization 
        var modifiedResponse = this.transformResponse(dsResponse, dsRequest, data);
        // handle failing to return the response - assume it's been modified in place
        dsResponse = modifiedResponse || dsResponse;
        // convert startRow, endRow and totalRows from string to integer if necessary, and
        // provide defaults if the values are senseless
        dsResponse.startRow = this._parseNumber(dsResponse.startRow, 0);
        var endRow = dsResponse.endRow;
        if (endRow == null) {
            if (dsResponse.status < 0) endRow = 0;
            else if (isc.isAn.Array(dsResponse.data)) endRow = dsResponse.data.length;
            else endRow = 1;
        }
        
        dsResponse.endRow = this._parseNumber(endRow);
        dsResponse.totalRows = this._parseNumber(dsResponse.totalRows, dsResponse.endRow);

        // if cacheAllData:false and autoCacheAllData:true and this fetch response includes
        // all posible records, switch cacheAllData on and proceed
        // Note 'downloadResult' requests bypass the cache-data so don't allow these to toggle
        // this.cacheAllData
        if (!this.cacheAllData && this.autoCacheAllData &&
            dsRequest.downloadResult != true && 
            dsResponse.operationType == "fetch" && 
            (dsRequest.data == null || isc.isAn.emptyObject(dsRequest.data)) &&
            dsResponse.startRow == 0 && dsResponse.endRow == dsResponse.totalRows)
        {
            if (this.logIsInfoEnabled("cacheAllData")) {
                this.logInfo("Response: switching on cacheAllData", "cacheAllData");
            }
            this.cacheAllData = true;
            this._autoCacheAllData_timestamp = isc.timestamp();
        }

        //>Offline cache responses for subsequent offline use.  Do this after the response is
        // known to be data-provider agnostic, that is, dsResponse.data is an Array of Object
        // and not any provider-specific format.  This allows custom data protocols to use the
        // offline system transparently, and skips the maximum amount of work when using
        // offline response (eg, no parsing WSDL responses again)
        if (this.useOfflineStorage && dsResponse.status == 0 && !this.clientOnly) {
            isc.DataSource.cacheResponse(dsRequest, dsResponse);
            // It's a waste of time to re-stash a response that came from Offline (and it also
            // leads to a misleading timestamp on the response - it effectively becomes a 
            // last-accessed timestamp rather than a stored timestamp)
            if (isc.Offline && !dsResponse.fromOfflineCache) {
                if (dsRequest.unconvertedDSRequest) {
                    isc.Offline.storeResponse(dsRequest.unconvertedDSRequest, dsResponse);
                } else {
                    isc.Offline.storeResponse(dsRequest, dsResponse);
                }
            }
        }
        
        // HACK: used by DBCs to tell if their data is in offline mode
        if (dsRequest && dsRequest.resultSet) {
            if (dsResponse.status == isc.RPCResponse.STATUS_OFFLINE) {
                dsRequest.resultSet._offline = true;
            } else {
                dsRequest.resultSet._offline = false;
            }
        } else if (dsRequest && dsRequest.resultTree) {
            if (dsResponse.status == isc.RPCResponse.STATUS_OFFLINE) {
                dsRequest.resultTree._offline = true;
            } else {
                dsRequest.resultTree._offline = false;
            }
        }
        //<Offline

        if (dsResponse.relatedUpdates != null) {
            for (var i = 0; i < dsResponse.relatedUpdates.length; i++) {
                if (dsResponse.relatedUpdates[i].operationType == null) {
                    dsResponse.relatedUpdates[i].operationType = dsResponse.operationType;
                }

                isc.DS.get(dsResponse.relatedUpdates[i].dataSource).updateCaches(dsResponse.relatedUpdates[i], null);
            }
        }

        this.fireResponseCallbacks(dsResponse, dsRequest, rpcResponse, rpcRequest);
    },

    fireResponseCallbacks : function (dsResponse, dsRequest, rpcResponse, rpcRequest) {
    
        if (!dsResponse.clientContext) dsResponse.clientContext = {};

        if (dsResponse.status >= 0) {
            // on success, notify ResultSets
            isc.DataSource.handleUpdate(dsResponse, dsRequest);
        } else if (!dsRequest.willHandleError) {
            isc.RPCManager._handleError(dsResponse, dsRequest);
        }

        // fire callbacks:
                        // passed directly to a DataSource method, eg ds.fetchData()
        var callbacks = [dsRequest._dsCallback, 
                        // secondary convenience callback, see ActionMethods::buildRequest()
                         dsRequest.afterFlowCallback],
            firedCallbacks = [];
            
        for (var i = 0; i < callbacks.length; i++) {

            var callback = callbacks[i];

            // paranoid duplicate callback check
            if (firedCallbacks.contains(callback)) {
                this.logWarn("Suppressed duplicate callback: " + callback);
                continue;
            }
            
            var callbackResult = this.fireCallback(callback, "dsResponse,data,dsRequest", 
                                                    [dsResponse,dsResponse.data,dsRequest]);
            
                                                        
			// the primary callback, which completes the basic dataSource flow, can stop
            // further processing by returning false.  This is intended to allow flow code to
            // signal fundamental errors that high-level user code isn't expected to handle.
			if (rpcRequest && rpcRequest.willHandleError && callbackResult === false) {
				//>DEBUG
				this.logDebug("performOperationReply: Further processing cancelled by callback");
				//<DEBUG
				break;
			}

            // If transaction has been suspended, stop processing callbacks
            // Can happen on EG login failure
            
            if (rpcResponse) {
                var transaction = isc.RPCManager.getTransaction(rpcResponse.transactionNum);            
                if (transaction && transaction.suspended) return;
            }
		}

    },

    // if argument is a string, try parsing it as a number and return parsed value as long as
    // it's not NaN
    _parseNumber : function (number, defaultValue) {
        if (number == null) return defaultValue;
        if (!isc.isA.String(number)) return number;
        var parsedNum = parseInt(number);
        if (isNaN(parsedNum)) return defaultValue != null ? defaultValue : number;
        else return parsedNum;
    },
    
    //> @method dataSource.transformResponse() [A]
    // Modify the DSResponse object derived from the response returned from the 
    // +link{dataSource.dataURL,dataURL}.
    // <P>
    // This is an override point that makes it possible to use DataSource features such as
    // paging with web services that support such features, by allowing you to fill in metadata
    // fields in the DSResponse object (such as +link{attr:DSResponse.startRow}) based on
    // service-specific metadata fields contained in the service's response.
    // <P>
    // The DSResponse passed to this method already has +link{attr:dsResponse.data}, which is
    // derived differently depending on the +link{dataSource.dataFormat,dataFormat} setting:
    // <ul>
    // <li> <code>dataFormat:"xml"</code> : either the
    // +link{attr:operationBinding.recordXPath,recordXPath} or 
    // +link{attr:operationBinding.recordName,recordName} is used to select the XML elements
    // that represent DataSource records.  The selected XML elements are passed to
    // +link{method:DataSource.recordsFromXML()}, which transforms the XML elements to typed
    // JavaScript data using the DataSource as a schema.
    // <li> <code>dataFormat:"json"</code> : the
    // +link{attr:operationBinding.recordXPath,recordXPath}, if specified, is used to select
    // records from the returned JSON data via +link{XMLTools.selectObjects()}.  
    // +link{DataSourceField.valueXPath} is used to derive correctly typed field values.
    // <li> <code>dataFormat:"custom"</code> : <code>dsResponse.data</code> is the raw response
    // in String form.  It must be parsed into an Array of Objects for subsequent processing to
    // work.
    // </ul> 
    // <P>
    // In addition to <code>dsResponse.data</code>, +link{attr:dsResponse.status} is defaulted
    // to 0 (indicating no error), and +link{attr:dsResponse.startRow} is assumed to be zero,
    // with +link{attr:dsResponse.endRow,endRow} and +link{attr:dsResponse.totalRows,totalRows}
    // both set to <code>dsResponse.data.length - 1</code>, that is, the returned data is
    // assumed to be all records that matched the filter criteria.
    // <P>
    // Examples of using this API include:
    // <ul>
    // <li> setting +link{attr:dsResponse.startRow,startRow}, 
    //      +link{attr:dsResponse.endRow,endRow} and +link{attr:dsResponse.totalRows,totalRows}
    //      to allow paging for a service that supports it.  For example, if an XML service
    //      returns a "resultRow" tag that contained the row number of the first row of the
    //      returned results:<pre>
    //    dsResponse.startRow = 
    //        isc.XMLTools.selectNumber(xmlData, "//resultRow");
    // </pre>
    // <li> setting +link{attr:dsResponse.status} to recognized ISC error values based on
    //      service-specific errors, in order to trigger standard ISC error handling.  For
    //      example, setting <code>dsResponse.status</code> to
    //      +link{rpcResponse.STATUS_VALIDATION_ERROR} and filling in
    //      +link{attr:dsResponse.errors} in order to cause validation errors to be shown in
    //      forms and grids.
    // <li> for services that either do not return cache update data, or return partial data,
    //      using +link{attr:dsRequest.oldValues} to create cache update data (whether this is
    //      appropriate is application-specific), or setting
    //      +link{attr:dsResponse.invalidateCache}.
    // </ul>
    // <P>
    // <span STYLE="color:red">NOTE:</span> this method is NOT an appropriate time to call
    // methods on visual components such as grids, initiate new DSRequests or RPCRequests, or
    // in general do anything other than fill in fields on the DSResponse based on data that is
    // already available.  Any actions that need to be taken as a result of the web
    // service response should be implemented exactly as for a DataSource where
    // <code>transformResponse()</code> has not been overridden, that is, use the callback
    // passed to high-level methods such as 
    // +link{group:dataBoundComponentMethods,<code>grid.fetchData()</code>}, and do error
    // handling via either +link{DataSource.handleError()} or by setting
    // +link{RPCRequest.willHandleError,willHandleError}.
    //
    // @param dsResponse (DSResponse) default DSResponse derived from the response data
    // @param dsRequest (DSRequest) DSRequest object that initiated this request
    // @param data (XMLDocument or JSON) XML document or JSON objects returned by the web
    //                                   service
    // @return (DSResponse) response derived 
    // @visibility xmlBinding
    // @example xmlServerValidationErrors
    //<
    
    transformResponse : function (dsResponse, dsRequest, data) {
        return dsResponse;
    },

    // get the value of a field of a record from this DataSource.  Used for dynamically
    // fetching field values from XML elements
    getFieldValue : function (record, fieldName, field) {
        var value = isc.xml.getFieldValue(record, fieldName, field, this, this.xmlNamespaces);
        if (!field.getFieldValue) return value;

        if (!isc.isA.Function(field.getFieldValue)) {
    	    isc.Func.replaceWithMethod(field, "getFieldValue", "record,value,field,fieldName");
        }

        return field.getFieldValue(record, value, field, fieldName);
    },

    _singleArray : [],
    // validate a value for a field of simple type
    validateFieldValue : function (field, value) {
        if (!isc.Validator) return value; // forms module not loaded

        var validators = field.validators;
        if (!validators) return value;

        // normalize to an Array if necessary, avoiding an allocation   
        if (!isc.isAn.Array(validators)) {
            this._singleArray[0] = validators;
            validators = this._singleArray;
        }

        var origValue = value; // purely for log below

        for (var i = 0; i < validators.length; i++) {
            var validator = validators[i];
            var result = isc.Validator.processValidator(field, validator, value, null, null);
            if (!result) {
                this.logWarn(this.ID + "." + field.name + 
                             ": value: " + this.echoLeaf(value) +
                             " failed on validator: " + this.echo(validator));
                // NOTE: the value we return is the value as transformed by what validators did
                // not fail.  Hence, having originally been a String, it may now be an integer,
                // but not in the correct range.
                return value;
            }
            var undef;
            if (validator.resultingValue !== undef) {
                value = validator.resultingValue;
                validator.resultingValue = undef;
            }
			// if the validator wants us to stop if it wasn't passed, break out of the
            // validators loop
			if (!result && validator.stopIfFalse) break;
        }

        //this.logWarn("At field: " + this.ID + "." + field.name + 
        //             " validating value: " + this.echoLeaf(origValue) +
        //             " with validators: " + this.echo(validators) + 
        //             " produced result: " + this.echo(value));

        this._singleArray.length = 0;
        return value;
    },

    // Give a standard or advanced criteria object, return the complete set of
    // field names referenced.
    getCriteriaFields : function (criteria) {
        return isc.DS.getCriteriaFields(criteria, this);
    },

    // Drill into advancedCriteria object extracting fields
    _getAdvancedCriteriaFields : function (criterion, fieldNames) {
        return isc.DS._getAdvancedCriteriaFields(criterion, fieldNames);
    },

    // DataSource operation methods
    // --------------------------------------------------------------------------------------------

    // NOTE: concurrent editing: 
    //   - for updates and deletes we'll need the oldValues to detect changes in the target
    //     record
    //   - oldValues will be passed as requestProperties.oldValues 
    //   - oldValues will remain optional since many datasets are single-user
    //     - NOTE however: even single user web applications can be logged into simultaneously
    //       from two locations (eg email)
    
    //> @method dataSource.fetchRecord()
    // Fetch a single record from the DataSource by +link{dataSourceField.primaryKey,primary key}.
    // This simply calls +link{fetchData()} after creating +link{Criteria} that contain the primary
    // key field and value.
    // <p>
    // If you call this method on a DataSource with a composite primary key - ie, one with 
    // multiple primaryKey fields - this method returns the first record where the first 
    // defined primary field matches the supplied pkValue; this may or may not be meaningful, 
    // depending on your use case.  Generally, for DataSources with composite keys, it makes
    // more sense to use <code>fetchData()</code> directly, rather than this convenience 
    // method.
    //
    // @param pkValue (any) value for the field marked
    //                      +link{dataSourceField.primaryKey,primaryKey}:true in this DataSource
    //                      (or the first field so marked if there is more than one)
    // @param [callback] (DSCallback)  callback to invoke on completion
    // @param [requestProperties] (DSRequest Properties) additional properties to set on the
    //                                                   DSRequest that will be issued
    // @visibility external
    //<
    fetchRecord : function (pkValue, callback, requestProperties) {
        var criteria = {},
            pkField = this.getPrimaryKeyField();
        if (pkField == null) {
            this.logWarn("This datasource has no primary key field. Ignoring fetchRecord call");
            return;
        }
        var pkFieldName = pkField.name;
            
        // handle being passed criteria rather than a single pkField value
        var undefined;
        if (isc.isAn.Object(pkValue) && pkValue[pkFieldName] !== undefined) {
            criteria = pkValue
        } else {
            criteria[pkFieldName] = pkValue
        }
        return this.fetchData(criteria, callback, requestProperties);
    },

     

    //> @class Callbacks
    // This object cannot be used; it exists for documentation purposes only as a place to put
    // documentation for callback methods, such as the callback for +link{Callbacks.DSCallback,DataSource.fetchData()}.
    //
    // @visibility external
    //<
    
    //> @method Callbacks.DSCallback
    // Callback fired when DataSource methods that send DSRequests complete (such as +link{DataSource.fetchData()}).
    //
    // @param dsResponse (DSResponse) a +link{class:DSResponse} instance with metadata about the returned data
    // @param data (Object) data returned to satisfy the DataSource request. See the 
    //        +link{group:dataSourceOperations,DataSource operations} topic for expected results
    //        for each type of DataSource operation
    // @param dsRequest (DSRequest) the +link{class:DSRequest} that was sent. You can use
    //        +link{attr:dsRequest.clientContext} to track state during the server turnaround.
    //
    // @visibility external
    //<
    
    //> @method dataSource.fetchData()
    // Perform a "fetch" DataSource operation against this DataSource, sending search criteria
    // and retrieving matching records.
    // <P>
    // <b>NOTE:</b> do not attempt to override this method to create a custom DataSource.  For
    // a server-side custom DataSource, use the +link{DataSource.serverConstructor} attribute,
    // and the +explorerExample{customDataSource,Custom DataSource samples}.  For a
    // client-side custom DataSource, see +link{dataProtocol,dataProtocol:"custom"}.
    // <P>
    // In contrast to +link{listGrid.fetchData()}, which creates a +link{ResultSet} to manage
    // the returned data, calling <code>dataSource.fetchData()</code> provides the returned
    // data in the callback as a 
    // <var class="smartclient">simple JavaScript Array of JavaScript Objects.</var>
    // <var class="smartgwt">RecordList or simple Array of Record objects.</var>  Calling
    // <code>dataSource.fetchData()</code> does not automatically update any visual components or
    // caches: code in the callback passed to <code>fetchData()</code> decides what to do with
    // the returned data. 
    // <P>
    // For example, given a ListGrid "myGrid" and a DataSource "employees", the following code
    // would populate "myGrid" with data fetched from the DataSource:
    // <pre>
    //    <var class="smartclient">isc.DataSource.get("employees").fetchData(null, "myGrid.setData(data)");</var>
    //    <var class="smartgwt">DataSource.get("employees").fetchData(null, new DSCallback() {
    //        public void execute(DSResponse response, Object rawData, DSRequest request) {
    //            myGrid.setData(response.getData());
    //        }
    //    });</var>
    // </pre>
    // Unlike calling <code>myGrid.fetchData()</code>, which creates a +link{ResultSet}, the
    // data provided to the grid is "disconnected" data, unmanaged by SmartClient's databinding
    // facilities and safe to directly modify.  This is useful when, for example, a ListGrid is
    // being used as a more sophisticated version of HTML's multi-select component.
    // <P>
    // Disconnected datasets may be used to populate various visual components.  For example,
    // while an individual FormItem can be configured to fetch
    // +link{formItem.valueMap,valueMap} options from a DataSource via the
    // +link{formItem.optionDataSource,optionDataSource} property, the following code shows
    // storing a dataset to derive valueMaps from later:
    // <pre>
    //    <var class="smartclient">
    //    isc.DataSource.get("countries").fetchData(null, "window.countries = data");
    //
    //    ... later, a form is created dynamically ...
    //
    //    function showForm() {
    //       isc.DynamicForm.create({
    //           items : [
    //              { name:"country", title:"Pick Country",
    //                valueMap: window.countries.getValueMap("countryId", "countryName")
    //              },
    //       ...
    //    </var>
    //    <var class="smartgwt">
    //      // Assume GlobalStore.allCountries is a public static variable of type RecordList 
    //      DataSource.get("countries").fetchData(null, new DSCallback(){
    //         public void execute(DSResponse response, Object rawData, DSRequest request) {
    //            GlobalStore.allCountries = response.getDataAsRecordList();
    //         }
    //      });
    //      ... later, while a DynamicForm is being created ...    
    //      SelectItem select = new SelectItem("country", "Pick Country");
    //      Map valueMap = GlobalStore.countries.getValueMap("countryId", "countryName");
    //      myItem.setValueMap(new LinkedHashMap(valueMap));
    //    </var>
    // </pre>
    // <P>
    // You can also create a ResultSet from the data retrieved from <code>fetchData()</code>,
    // like so:
    // <pre>
    //    <var class="smartclient">
    //    isc.DataSource.get("countries").fetchData(null,
    //        function (dsResponse, data) {
    //            isc.ResultSet.create({
    //                dataSource:"countries",
    //                allRows:data
    //            })
    //        }
    //    )
    //    </var>
    //    <var class="smartgwt">
    //    DataSource.get("countries").fetchData(null, new DSCallback() {
    //        public void execute(DSResponse response, Object rawData, DSRequest request) {
    //            ResultSet rs = new ResultSet(DataSource.get("countries"));
    //            rs.setAllRows(response.getData());
    //        }
    //    });
    //    </var>
    // </pre>
    // <P>
    // This gives you a dataset that supports client-side filtering (via
    // +link{resultSet.setCriteria,setCriteria()}), can provide 
    // +link{resultSet.getValueMap,filtered valueMaps}, will 
    // +link{ResultSet.disableCacheSync,automatically reflect updates} to the DataSource made via
    // other components, and can be re-used with multiple visual components.
    // <P>
    // See also +link{DataSource.getClientOnlyDataSource()} and +link{DataSource.cacheAllData} for
    // similar capabilities for dealing with smaller datasets entirely within the browser, or working
    // with modifiable caches representing subsets of the data available from a DataSource.
    // <P>
    // See also the server-side com.isomorphic.js.JSTranslater class in the
    // +docTreeLink{javaServerReference,Java Server Reference} for other, similar approaches
    // involving dumping data into the page during initial page load.  <b>Note:</b> care should
    // be taken when using this approach.  Large datasets degrade the basic performance of some
    // browsers, so use +link{pickList.optionDataSource,optionDataSource} and similar
    // facilities to manage datasets that may become very large.
    // <P>
    // <b>Data-Driven Visual Component Creation</b>
    // <P>
    // <var class="smartclient">
    // <code>DataSource.fetchData()</code> can also be used to create SmartClient components in
    // a data-driven way.  Many properties on SmartClient visual components are configured via
    // an Array of Objects - the same data format that <code>dataSource.fetchData()</code>
    // returns.  These include +link{listGrid.fields}, +link{tabSet.tabs},
    // +link{dynamicForm.items}, +link{facet.values} and even +link{dataSource.fields}.
    // <P>
    // For example, if you had a DataSource "myFormFields" whose fields included the
    // basic properties of +link{FormItem,FormItems} (name, title, type, etc), this example
    // code would create a form based on stored field definitions, loaded from the
    // "myFormFields" DataSource on the fly:
    // <pre>
    //    isc.DataSource.get("myFormFields").fetchData(null, 
    //        "isc.DynamicForm.create({ items:data })"
    //    )
    // </pre>
    // </var>
    // <var class="smartgwt">
    // <code>DataSource.fetchData()</code> can also be used to create Smart GWT components in a
    // data-driven way.  For example, if you had a DataSource "myGridFields" whose fields included the
    // basic properties of {@link com.smartgwt.client.widgets.grid.ListGridField} (name, title, type,
    // etc), this example code would create a form based on stored field definitions, loaded from the
    // "myFormFields" DataSource on the fly: 
    // <pre>    
    //   DataSource.get("myFormFields").fetchData(null, new DSCallback(){
    //       public void execute(DSResponse response, Object rawData, DSRequest request) {
    //          Record[] records = response.getData();
    //          ListGridField[] fields = new ListGridField[records.length];
    //          for (Record record in records) {
    //              ListGridField field = new ListGridField();
    //              field.setName(record.getAttribute("name"));
    //              field.setTitle(record.getAttribute("title"));
    //              field.setType(ListGridFieldType.valueOf(record.getAttribute("type")));
    //          }
    //          ListGrid grid = new ListGrid();
    //          grid.setFields(fields);
    //       }
    //   });
    // </pre>  
    // </var>
    // This capability to dynamically create visual components from dynamically fetched data
    // provides a foundation for creating interfaces that can be customized by end users.
    // See also the server-side API com.isomorphic.datasource.DataSource.addDynamicDSGenerator() for
    // dynamically creating DataSources supporting all server-side DataSource features, and
    // +link{dataSource.inheritsFrom} for sharing field definitions across multiple DataSources.
    //
    //	@param	[criteria]          (Criteria)	  search criteria
    //	@param	[callback]          (DSCallback)  callback to invoke on completion
    //  @param  [requestProperties] (DSRequest Properties)   additional properties to set on
    //                                                       the DSRequest that will be issued
    // @group operations
    // @visibility external
    //<
    fetchData : function (criteria, callback, requestProperties) {
        this.performDSOperation("fetch", criteria, callback, requestProperties);
    },

    //> @method dataSource.filterData()
    // Perform a "fetch" DataSource operation against this DataSource, sending search criteria
    // and retrieving matching records.
    // <P>
    // This is identical to +link{fetchData()} except that
    // +link{dsRequest.textMatchStyle} is set to "substring" to cause case insensitive
    // substring matching (if the server respects this setting).
    //
    //	@param	[criteria]          (Criteria)	  search criteria
    //	@param	[callback]          (DSCallback)  callback to invoke on completion
    //  @param  [requestProperties] (DSRequest Properties)   additional properties to set on
    //                                                       the DSRequest that will be issued
    // @group operations
    // @visibility external
    //<
    filterData : function (criteria, callback, requestProperties) {
        if (!requestProperties) requestProperties = {};
        if (requestProperties.textMatchStyle == null) requestProperties.textMatchStyle = "substring";
        this.performDSOperation("fetch", criteria, callback, requestProperties);
    },

    //> @attr dataSource.emptyExportMessage (string : "You are attempting to export an empty dataset" : [IRW])
    // The message to display to the user if an export of client-side data is attempted, but 
    // data provided for export is null or an empty List.
    // @see dataSource.exportClientData
    // @visibility internal - for now
    //<
    emptyExportMessage: "You are attempting to export an empty dataset",

    //> @method dataSource.exportClientData()
    // Exports arbitrary client-side data, with client-side formatters applied, so is suitable 
    // for direct display to users.  This method can be used to export data formatted outside
    // of any kind of visual component.
    // <P>
    // If you do not specify an +link{operationBinding.operationId,operationId} in the 
    // <code>requestProperties</code> you pass to this method, it behaves exactly the same as
    // the +link{classMethod:DataSource.exportClientData,static classMethod} of the same name.
    // If you do specify an <code>operationId</code>, the framework expects your DataSource to
    // configure an +link{class:operationBinding} of +link{operationBinding.operationType,operationType}
    // "clientExport" with the same <code>operationId</code>.  The framework will then send the
    // <code>exportClientData</code> request via the ordinary +link{class:DSRequest} mechanism,
    // which allows you to use normal framework features in the client data export.  
    // <p>
    // For example, you could add a +link{group:DMI,DMI declaration} to your
    // <code>operationBinding</code>, which would allow you to write server-side code that
    // intervenes in the export process - for instance, by calling the
    // <code>getExportObject()</code> API to do something special with the export document,
    // like saving it to a database table or sending it to an email list.
    // <p>
    // When you use the specific <code>operationId</code> version of this API, both the 
    // SmartClient Server and server-side DataSources are required.
    // <P>
    // To export unformatted data, see +link{dataSource.exportData, exportData} which does
    // not include client-side formatters, but requires both the SmartClient server and the 
    // presence of server-side DataSources.
    // <P>
    // Note that field +link{DataSourceField.displayFormat,displayFormat} is honored for 
    // "date" and "datetime" fields when exporting direct to Excel; see the displayFormat
    // docs for details.
    //
    // @param data (Array of Record) Array of Records to export
    // @param requestProperties (DSRequest properties) Request properties for the export
    // @param callback (DSCallback) Optional callback.  Note that this is only applicable 
    // if you also specify +link{DSRequest.exportToClient,exportToClient}: false in the 
    // request properties
    // @visibility external
    //<
    exportClientData : function (data, requestProperties, callback) {
        if (requestProperties.operationId) {
            if (!this.operationBindings) {
                this.logWarn("exportClientData() was passed operationId '" +
                             requestProperties.operationId +
                             "', but this DataSource has no operationBindings.  Delegating to the " +
                             "static exportClientData() method");   
                delete requestProperties.operationId;
            }
            var binding, undef;
            for (var i = 0; i < this.operationBindings.length; i++) {
                if (this.operationBindings[i].operationId == requestProperties.operationId &&
                    this.operationBindings[i].operationType == "clientExport")
                {
                    binding = this.operationBindings[i];
                }
            }
            if (binding === undef) {
                this.logWarn("exportClientData() was passed operationId '" +
                             requestProperties.operationId +
                             "', but this DataSource has no operationBinding of type 'clientExport' " +
                             "with that name.  Delegating to the static exportClientData() method");   
                delete requestProperties.operationId;
            }
        }
        return isc.DataSource.exportClientData(data,requestProperties, callback, this);
	},

    //> @method dataSource.exportData()
    // Perform a "fetch" DataSource operation against this DataSource, sending search criteria,
    // retrieving matching records and exporting the results.  See 
    // +link{operationBinding.exportResults} or +link{dsRequest.exportResults} and for more
    // information.
    //
    //	@param	[criteria]          (Criteria)	  search criteria
    //  @param  [requestProperties] (DSRequest Properties)   additional properties to set on
    //                                                       the DSRequest that will be issued
    // @param [callback] (DSCallback)   callback to invoke on completion.  Note that this parameter
    //                                  only applies where +link{dsRequest.exportToClient} is 
    //                                  explicitly set to false, because  file downloads do not 
    //                                  provide ordinary SmartClient callbacks 
    // @group operations
    // @visibility external
    //<
    exportData : function (criteria, requestProperties, callback, component) {
        if (!requestProperties) requestProperties = {};

        if (this.canExport == false) {
        	// exporting is disabled at the DS level - warn and return
        	isc.logWarn("Exporting is disabled for this DataSource.  Set " + 
                "DataSource.canExport to true to enable it.");
        	return;
        }

        if (requestProperties.exportAs && requestProperties.exportAs.toLowerCase() == "json" &&
            requestProperties.exportToClient !== false)
        {
        	// json export not allowed via client-calls
        	isc.logWarn("Export in JSON format with download to client is not allowed from a " +
                        "client call - either specify exportToClient: false, or set " + 
                        "operationBinding.exportAs on your DataSource instead.  The server " +
                        "will reject this attempt.");
        }

        if (requestProperties.textMatchStyle == null) requestProperties.textMatchStyle = "substring";

		var parameters = {};
        parameters.exportResults = true;
        parameters.exportAs = requestProperties.exportAs || "csv"; 
        parameters.exportDelimiter = requestProperties.exportDelimiter || ","; 
        parameters.exportTitleSeparatorChar = requestProperties.exportTitleSeparatorChar || ""; 
        parameters.exportFilename = requestProperties.exportFilename || "Results." + 
        	(parameters.exportAs == "ooxml" ? "xlsx" : parameters.exportAs);
        requestProperties.exportFilename = parameters.exportFilename;
        parameters.exportPath = requestProperties.exportPath;
        parameters.exportDisplay = requestProperties.exportDisplay || "download";
        parameters.lineBreakStyle = requestProperties.lineBreakStyle || "default"; 
        parameters.exportFields = this.getExportableDSFields(
            requestProperties.exportFields  || this.getVisibleDSFields(), component
        );
        parameters.exportHeader = requestProperties.exportHeader;
        parameters.exportHeaderless = requestProperties.exportHeaderless;
        parameters.exportFooter = requestProperties.exportFooter;
        parameters.exportFieldTitles = requestProperties.exportFieldTitles;
        parameters.exportDatesAsFormattedString = requestProperties.exportDatesAsFormattedString;

        if (!parameters.exportFieldTitles) {
            var wkFields = parameters.exportFields;
            var exportFieldTitles = {};
            for (var i = 0; i < wkFields.length; i++) {
                var field = wkFields[i];
                var fieldName;
                if (isc.isA.String(field)) {
                    fieldName = field;
                    field = this.getField(fieldName);
                }
                if (field) {
                    if (field.hidden) continue;
                   exportFieldTitles[field.name] = field.exportTitle || field.title;
                } else {
                   exportFieldTitles[fieldName] = fieldName;
                }
            }
            parameters.exportFieldTitles = exportFieldTitles;
        }

        parameters.exportHeaderSpans = requestProperties.exportHeaderSpans;
        parameters.exportOtherFields = requestProperties.exportOtherFields;

        requestProperties.downloadResult = (requestProperties.exportToClient !== false);
        requestProperties.downloadToNewWindow = requestProperties.exportDisplay == "window" 
        	? true : false;

		if (requestProperties.downloadToNewWindow) {
			if (parameters.exportFilename.endsWith(".xml") && parameters.exportAs != "xml") {
				parameters.exportFilename = parameters.exportFilename + ".txt";
			}
            requestProperties.download_filename = requestProperties.exportFilename;
            criteria = criteria || {};
            criteria.download_filename = requestProperties.download_filename;
        }
        requestProperties.showPrompt = false;
		requestProperties.parameters = parameters;

        this.performDSOperation("fetch", criteria, callback, requestProperties);
    },

	getVisibleDSFields : function () {
        // pass up only visible fields
        var vFields = [];
        var fields = this.fields;
        // Internally, the fields collection is held as an index, not an array, so convert
        if (!isc.isAn.Array(fields)) {
            fields = [];
            for (var key in this.fields) {
                fields.add(this.fields[key]);
            }
        }
        for (var i = 0; i < fields.length; i++) {
            var field = fields.get(i);
            if (!field.hidden) vFields.add(field.name);
        }
        return vFields;
	},

    getExportableDSFields : function (fields, component) {
        // pass up only those fields that can be exported
        var vFields = [];
        if (this.canExport) {
            for (var i = 0; i < fields.length; i++) {
                var field = this.getField(fields[i]);
                if (!field && component && component.getField) {
                    field = component.getField(fields[i]);
                    if (field && field.includeFrom && field.canExport != false) {
                        vFields.add(fields[i]);
                    }
                    continue;
                }
                if (field && field.canExport != false) 
                    vFields.add(field.name);
            }
        }
        return vFields;
	},

    // undoc'd ability to disable exports at the DS level
    canExport: true,

    //> @method Callbacks.Function
    // Generic callback interface.
    //
    // @visibility external
    //<

    //> @method Callbacks.ClientOnlyDataSourceCallback
    // Generic callback interface.
    //
    // @param clientOnly (DataSource) Client only Data Source.
    // @visibility external
    //<
    
    //> @method dataSource.getClientOnlyDataSource()
    // Produces a clientOnly "copy" of a particular subset of data from a normal 
    // DataSource, via calling fetchData() to fetch matching rows, and constructing 
    // a clientOnly DataSource that +link{inheritsFrom} the original DataSource.
    // <P>
    // This clientOnly "copy" can be useful in situations where you want to allow 
    // a series of local changes without immediately committing to the server.  
    // See also +link{ListGrid.autoSaveEdits} for more fine-grained tracking of 
    // edits (eg, special styling for uncommitted changes).
    // <P>
    // The new DataSource is returned via the "callback" argument.  If +link{cacheAllData} is
    // enabled and +link{hasAllData()} returns true, the new DataSource is synchronously
    // returned as the result of the method.  In this case, if a callback was passed, it also
    // is executed synchronously.
    //
    // @param criteria (Criteria) The criteria for the clientOnly DS
    // @param callback (ClientOnlyDataSourceCallback) The callback to fire passing the clientOnly DS
    // @param [requestProperties] (DSRequest Properties) optional properties to pass through to the DSRequest
    // @param [dataSourceProperties] (DataSource Properties) optional properties to pass 
    //                                                       through to the clientOnly DS
    // 
    // @visibility external
    //<
    getClientOnlyDataSource : function (criteria, callback, requestProperties, dataSourceProperties) {
        var _criteria = criteria,
            _callback = callback,
            parentDS = this;

        if (this.cacheAllData && this.hasAllData()) {
            var ds = isc.DataSource.create({
                inheritsFrom: parentDS,
                clientOnly: true,
                useParentFieldOrder: true,
                testData: this.cacheResultSet.getAllRows()
                }, dataSourceProperties)
            ;
            parentDS.fireCallback(_callback, "dataSource", [ds]);
            return ds;
        } else {
            this.fetchData(_criteria, 
                function (dsResponse, data) {
                    var rows = dsResponse.totalRows;
                    parentDS.fetchData(_criteria,
                        function (dsResponse, data) {
                            var ds = isc.DataSource.create({
                                inheritsFrom: parentDS,
                                clientOnly: true,
                                useParentFieldOrder: true,
                                testData: data
                                }, dataSourceProperties)
                            ;
                            parentDS.fireCallback(_callback, "dataSource", [ds]);
                        }, isc.addProperties({}, requestProperties, {startRow:0, endRow: rows})
                    )
                }, isc.addProperties({}, requestProperties, { startRow:0, endRow:0 })
            );
        }
    },

    //> @method dataSource.addData()
    // Perform an "add" DataSource operation against this DataSource, to create a new DataSource
    // record.
    // <P>
    // <b>NOTE:</b> do not use this method to populate a +link{clientOnly} DataSource.  Set
    // +link{cacheData} instead.
    //
    //	@param	newRecord           (Record)      new record
    //	@param	[callback]          (DSCallback)  callback to invoke on completion
    //  @param  [requestProperties] (DSRequest Properties)   additional properties to set on
    //                                                       the DSRequest that will be issued
    // @group operations
    // @visibility external
    //<
    addData : function (newRecord, callback, requestProperties) {
        if (isc.Offline && isc.Offline.isOffline() && this.contactsServer()) {
            isc.logWarn("Data cannot be saved because you are not online");
            return;
        }
        this.performDSOperation("add", newRecord, callback, requestProperties);
    },

    //> @method dataSource.updateData()
    // Perform an "update" DataSource operation against this DataSource, to update values in an
    // existing DataSource record.
    //
    //	@param	updatedRecord       (Record Properties)      updated record
    //	@param	[callback]          (DSCallback)  callback to invoke on completion
    //  @param  [requestProperties] (DSRequest Properties)   additional properties to set on
    //                                                       the DSRequest that will be issued
    // @group operations
    // @visibility external
    //<
    updateData : function (updatedRecord, callback, requestProperties) {
        if (isc.Offline && isc.Offline.isOffline() && this.contactsServer()) {
            isc.logWarn("Data cannot be saved because you are not online");
            return;
        }
        this.performDSOperation("update", updatedRecord, callback, requestProperties);
    },

    //> @method dataSource.removeData()
    // Perform a "remove" DataSource operation against this DataSource, to delete an existing
    // DataSource record.
    //
    //	@param	data                (Record PrimaryKeys) primary key values of record to delete, 
    //                                            (or complete record)
    //	@param	[callback]          (DSCallback)  callback to invoke on completion
    //  @param  [requestProperties] (DSRequest Properties)   additional properties to set on
    //                                                       the DSRequest that will be issued
    // @group operations
    // @visibility external
    //<
    removeData : function (recordKeys, callback, requestProperties) {
        if (isc.Offline && isc.Offline.isOffline() && this.contactsServer()) {
            isc.logWarn("Data cannot be saved because you are not online");
            return;
        }
        // if passed a full record, trim it down to primaryKeys.
        // This will avoid problems where additional properties such as _selected etc 
        // are present on the record
        
        var keyFields = this.getPrimaryKeyFields(),
            recordKeys = isc.applyMask(recordKeys, keyFields);
        this.performDSOperation("remove", recordKeys, callback, requestProperties);
    },
    
    contactsServer : function () {
        return !this.clientOnly && this.dataProtocol != "clientCustom"
    },

    //> @method dataSource.validateData()
    // Contacts the server to run server-side validation on a DSRequest and either returns
    // +link{dsResponse.errors} validation errors or a +link{dsResponse.status} code of 0.
    // <p>
    // A "validate" dsRequest is effectively always +link{rpcRequest.willHandleError}:true.
    // It is a normal condition for a "validate" DSResponse to have validation errors and
    // the response will never go to system-wide handling for unexpected errors
    // (+link{RPCManager.handleError}).
    //
    // @param values (Record) record values to validate
    // @param [callback] (DSCallback) callback to invoke on completion
    // @param [requestProperties] (DSRequest Properties) additional properties to set on
    //                                                   the DSRequest that will be issued
    //
    // @group operations
    // @visibility external
    //<
    validateData : function (values, callback, requestProperties) {
        if (!requestProperties) requestProperties = {};
        // Force willHandleError: true on request and default value for validationMode
        requestProperties = isc.addProperties(requestProperties, {willHandleError: true});
        if (requestProperties.validationMode == null) requestProperties.validationMode = "full";

        return this.performDSOperation("validate", values, callback, requestProperties);
    },

    //> @method dataSource.performCustomOperation()
    // Invoke an operation declared with +link{operationBinding.operationType} "custom".
    // <P>
    // <b>This is a rarely used API.</b>  If the operation you are performing can be thought of
    // as one of the standard "CRUD" +link{type:DSOperationType,operation types}, declare it
    // with a CRUD operationType.  For example, if your operation updates a record, declare it
    // with operationType "update" and invoke it via +link{updateData()} - this will cause
    // +link{ResultSet,cache sync} to work correctly.
    // <P>
    // In particular:
    // <ul>
    // <li> do not use this API just because you need to add additional server-side logic to a
    // CRUD operation (+link{DMI} allows this)
    // <li> do not use this API to implement variants of core CRUD operations
    // (+link{dsRequest.operationId} is the correct way to do this)
    // <li> do not use this API just because an operation affects more than one record.  Most
    // kinds of multi-record operations should use +link{RPCManager.startQueue(),queuing}.
    // <li> do not use this API just because you are calling a stored procedure in SQL - if the
    // stored procedure performs some kind of CRUD operation on the records of this DataSource,
    // use a standard CRUD operationType
    // </ul>
    // <P>
    // Instead, the specific purpose of this API is to bypass all checks and side effects that
    // normally occur for CRUD operations, for example, that a "fetch" requires valid Criteria
    // or that an "update" or "remove" operation contains a valid primary key, or that an "add"
    // operation returns the newly added record.  <code>performCustomOperation</code> allows
    // you to pass an arbitrary Record to the server, act on it with custom code, and return
    // arbitray results or even no results.
    // <P>
    // The "data" parameter becomes +link{dsRequest.data}.  With the SmartClient Server
    // Framework, the data is accessible server-side via DSRequest.getValues() and in Velocity
    // templates (such as &lt;customSQL&gt;) as $values.
    // <P>
    // Note that with SQLDataSource, <code>performCustomOperation</code> must be used if you
    // plan to have a &lt;customSQL&gt; tag in your operationBinding that will execute SQL
    // operations other than SELECT, UPDATE, INSERT, DELETE (such as creating a new table).
    // By declaring +link{operationBinding.operationType} "custom" in your .ds.xml file, all
    // checks related to normal CRUD operations will be skipped and your &lt;customSQL&gt;
    // can do arbitrary things.
    //
    //  @param  operationId         (String)                 the operation ID
    //	@param	[data]              (Record Properties)      data to pass to the server.
    //	@param	[callback]          (DSCallback)  callback to invoke on completion
    //  @param  [requestProperties] (DSRequest Properties)   additional properties to set on
    //                                                       the DSRequest that will be issued
    // @group operations
    // @visibility external
    //<
    performCustomOperation : function (operationId, data, callback, requestProperties) {
        if (!requestProperties) requestProperties = {};
        isc.addProperties(requestProperties, {operationId: operationId});
        this.performDSOperation("custom", data, callback, requestProperties);
    },

    performClientExportOperation : function (operationId, data, callback, requestProperties) {
        if (!requestProperties) requestProperties = {};
        isc.addProperties(requestProperties, {operationId: operationId});
        this.performDSOperation("clientExport", data, callback, requestProperties);
    },

    

    _getNextRequestId : function () {
        // ID can be completely arbitrary as long as it's unique, but lets use a format like
        // "SupplyItem_request12" to simplify debugging.
        if (!this._requestIdTemplate) this._requestIdTemplate = [this.getID(), "_request"];
        this._requestIdTemplate[2] = isc.DataSource._getNextRequestId();
        return this._requestIdTemplate.join(isc.emptyString);
    },
    
    // In order to avoid IE9 leaks we want to use strict JSON parsing (via JSON.parse)
    // by default for iscServer dataSources
    
    shouldUseStrictJSON : function (request) {
    	if (request && request.useStrictJSON != null) return request.useStrictJSON;
    	if (this.useStrictJSON != null) return this.useStrictJSON;
    	
    	if (isc.Browser.isIE && isc.Browser.version>=9
    		 && isc.RPCManager.useJSONParse_IE9 &&
    		this.dataFormat == "iscServer") 
    	{
    		return true;
    	}
    	return null;		
    },
    shouldFallbackToEval : function (request) {
    	if (request == null) request = {};
    	if (request.fallbackToEval != null) return request.fallbackToEval;
    	if (request.useStrictJSON || this.useStrictJSON) return false;
    	return (isc.Browser.isIE && isc.Browser.version>=9 
    			&& isc.RPCManager.useJSONParse_IE9 && this.dataFormat == "iscServer");
    },
    
    performDSOperation : function (operationType, data, callback, requestProperties) {
        if (isc._traceMarkers) arguments.__this = this;

        // form a dsRequest
        var dsRequest = isc.addProperties({
            operationType : operationType,
            dataSource : this.ID,
            data : data,
            callback : callback,
            requestId : this._getNextRequestId(),
            // support useStrictJSON at the DS and at the
            // request level.
            
            useStrictJSON : this.shouldUseStrictJSON(requestProperties),
            fallbackToEval : this.shouldFallbackToEval(requestProperties)
        }, requestProperties);
        
        isc.DataSource.recordTimingData(dsRequest, "UI event to DSRequest creation", "start", isc.EH._setThreadTimeStamp);
        isc.DataSource.recordTimingData(dsRequest, "UI event to DSRequest creation", "end");
        isc.DataSource.recordTimingData(dsRequest, "performDSOperation", "start");
        dsRequest.lastClientEventThreadCode = isc.EH._thread;
        
        var ev = isc.EH.getLastEvent();
        if (dsRequest.sortBy != null) {
            if (!isc.isAn.Array(dsRequest.sortBy)) dsRequest.sortBy = [dsRequest.sortBy];

            if (isc.isAn.Object(dsRequest.sortBy[0])) {
                dsRequest.sortBy = isc.DS.getSortBy(dsRequest.sortBy);
            }

            for (var i = 0; i < dsRequest.sortBy.length; i++) {
                var sortByProp = dsRequest.sortBy[i];
                if (!isc.isA.String(sortByProp)) continue;
                // get the sortField, which might be prefixed with -
                var field = this.getField(sortByProp.charAt(0) == "-" ? 
                                sortByProp.substring(1) : sortByProp);
                // don't request the sort if the field is canSortClientOnly
                if (field && field.canSortClientOnly) dsRequest.sortBy[i] = null;
            }
            dsRequest.sortBy.removeEmpty();
            if (dsRequest.sortBy.length == 0) delete dsRequest.sortBy;
        }
        if (this.sendParentNode == false) dsRequest.parentNode = null;

        return this.sendDSRequest(dsRequest);
    },

    sendDSRequest : function (dsRequest) {
        // provide default requestProperties for the operationBinding and the DataSource as a
        // whole
        isc.addDefaults(dsRequest, 
                        this.getOperationBinding(dsRequest.operationType).requestProperties);
        isc.addDefaults(dsRequest, this.requestProperties);

        // check for iscServer missing, and switch to clientOnly mode if necessary
        var dataFormat = this.getDataFormat(dsRequest);
        var protocol = this.getDataProtocol(dsRequest);
        if (dataFormat == "iscServer" && !(this.clientOnly || protocol == "clientCustom") &&
            !isc.hasOptionalModule("SCServer"))
        {
            if (this.dataURL == null && this.testFileName == null) {
                this.logError("DataSource: "+this.ID+
                          ": attempt to use DataSource of type iscServer without SmartClient Server option."+
                          " Please either set clientOnly: true for one-time fetch against"+
                          " dataURL/testFileName or upgrade to SmartClient Pro, Power or Enterprise");
                return;
            }
            // have dataURL or testFileName, default to clientOnly mode automatically
            this.logInfo("Switching to clientOnly - no SmartClient Server installed.");
            this.clientOnly = true;
        }
        
        if (dsRequest.bypassCache == null) {
            dsRequest.bypassCache = this.shouldBypassCache(dsRequest);
        }

        // default showPrompt:true according to DS setting
        if (dsRequest.showPrompt == null) {
            dsRequest.showPrompt = dsRequest.downloadResult ? false : this.showPrompt;
        }
        
        // if cacheAllData:false and autoCacheAllData:true and this request will fetch all data, 
        // switch cacheAllData on and proceed
        // Note 'downloadResult' requests bypass the cache-data so don't allow these to toggle
        // this.cacheAllData
        if (!this.cacheAllData && this.autoCacheAllData && 
        		dsRequest.downloadResult != true && 
                dsRequest.operationType == "fetch" && 
                dsRequest.startRow == null && dsRequest.endRow == null && 
                (dsRequest.data == null || isc.isAn.emptyObject(dsRequest.data)))
        {
            if (this.logIsInfoEnabled("cacheAllData")) {
                this.logInfo("sendDSRequest: switching on cacheAllData", "cacheAllData");
            }
            this.cacheAllData = true;
            this._autoCacheAllData_timestamp = isc.timestamp();
        }


		// If this is a clientOnly / cacheAllData dataSource and we don't yet have
		// a cache to fetch against, bail.
		// This method will kick off the one time fetch for the data as necessary and add
		// the method to the deferredRequests queue to be processed when that fetch returns.
        if (dsRequest.operationType == "fetch" && this.fetchingClientOnlyData(dsRequest)) return;

        if (this.logIsDebugEnabled()) {
            this.logDebug("Outbound DSRequest: " + this.echo(dsRequest));
        }
        // store off the dataSource layer callback as we will be passing a different callback
        // to the RPC layer
        dsRequest._dsCallback = dsRequest.callback;
        
        
        
        // Winnow the values object as follows:
        // - If dataSource.sparseUpdates is set, we remove values:
        //   - That are not present in this component's fields list
        //   - That are present but have an identical value in oldValues
        //   - But we keep all primaryKey fields
        // - If DataSource.noNullUpdates is set, we:
        //   - Remove properties from the values object where the value is null, even if it 
        //     isn't null in oldValues, if this is an add operation.
        //   - Replace the null value with one of the following, if this is an update operation:
        //     - The field's nullReplacementValue, if it has one
        //     - Otherwise the DataSource's nullStringValue, nullIntegerValue, nullFloatValue
        //       or nullDateValue, depending on the field's underlying base SimpleType
        var opType = dsRequest.operationType;
        if ((opType == "update" || opType == "add") && 
            (this.sparseUpdates || this.noNullUpdates)) 
        {
            isc.DataSource.recordTimingData(dsRequest, "Sparse updates processing", "start");
            // Work on a copy of the data - otherwise, we could trim away values that users 
            // are actually looking at, and a redraw between now and cache sync would make
            // them disappear...
            dsRequest.data = this._cloneValues(dsRequest.data, this);
            this._applySparseAndNoNullUpdates(dsRequest.data, dsRequest.oldValues, opType, this);
            isc.DataSource.recordTimingData(dsRequest, "Sparse updates processing", "end");
        }

        if (dataFormat == "iscServer") {
            this._storeCustomRequest(dsRequest);

            
            dsRequest.unconvertedDSRequest = isc.shallowClone(dsRequest);
            
            // This is a "reviver" function ultimately passed to JSON.parse
            // if useStrictJSON is true for this method.
            // Our built in version for iscServer format requests will 
            // parse dates from special formatted strings.
            // This will have no effect if useStrictJSON is not set on the request.
            dsRequest.jsonReviver = isc.DataSource.iscServer_strictJSONReviver;

            if (this.autoConvertRelativeDates == true) {
                // convert any relative dates in criteria into absolute dates so the server
                // doesn't need to know how to handle relative dates
                if (this.logIsInfoEnabled("relativeDates")) {
                    this.logInfo("Calling convertRelativeDates from sendDSRequest "+
                        "- data is\n\n"+isc.echoFull(data));
                }
                
                var data = this.convertRelativeDates(dsRequest.data);

                if (this.logIsInfoEnabled("relativeDates")) {
                    this.logInfo("Called convertRelativeDates from sendDSRequest "+
                        "- data is\n\n"+isc.echoFull(data));
                }

                dsRequest.data = data;
            }

            var transformedData = this.transformRequest(dsRequest);
            // correct the common error of returning the dsRequest itself incorrectly, which is
            // never right since the dsRequest contains various widgets and other data
            // inappropriate to send to the server.
            if (transformedData !== dsRequest) {
                data = dsRequest.data = transformedData;
            }

            // If this is a clientCustom operation we're done
            // (protocol can be set dynamically in the transformRequest method)
            // This implies the transformRequest implementation will have 
            // kicked off a request and we'll be notified via an explicit call to
            // processResponse() when new data is available.

            if (protocol == "clientCustom") {
                // _isServerRequest is installed when cacheAllData is active
                if (!this._isServerRequest) return;
                // only bail here for server requests - otherwise, use the cache as usual
                if (this._isServerRequest(dsRequest)) {
                    this.logInfo("sendDSRequest: clientCustom server request - returning", 
                        "cacheAllData"
                    );
                    return;
                } else {
                    this.logInfo("sendDSRequest: clientCustom request using client cache", 
                        "cacheAllData"
                    );
                    
                    //delete this._clientCustomRequests[dsRequest.requestId];
                }
            } else {
                // We now know this is not a client-custom operation - clear out the
                // request from the client-custom store so we don't leak requests.
                delete this._clientCustomRequests[dsRequest.requestId];
            }

            // detect whether this is a clientOnly request
            if ( dsRequest.shouldUseCache === false ||
                (dsRequest.shouldUseCache !== true) &&
                !this.clientOnly && 
                (!this.cacheAllData || dsRequest.downloadResult || dsRequest.cachingAllData || 
                 (dsRequest.operationType != null && dsRequest.operationType != "fetch") ||
                 (dsRequest.operationType == "fetch" && this.cacheAllOperationId &&
                     this.cacheAcrossOperationIds == false && 
                     dsRequest.operationId != this.cacheAllOperationId))) 
            {   
                // not client-only: this is a request to the SC server

                if (this.logIsInfoEnabled("cacheAllData") && dsRequest.cachingAllData) {
                    // special request from a cacheAllData DataSource to populate cache
                    this.logInfo("sendDSRequest: processing cacheAllData request", "cacheAllData");
                }

                this.addDefaultCriteria(dsRequest, this.getOperationBinding(dsRequest));
                this.applySendExtraFields(dsRequest);

                var operationBinding = this.getOperationBinding(dsRequest);
                if (operationBinding == null) operationBinding = {};
                var defaultParams = operationBinding.defaultParams || this.defaultParams;
                if (defaultParams) {
                    dsRequest.params = isc.addProperties({}, defaultParams, dsRequest.params);
                }
                data = dsRequest.data;

                return this.performSCServerOperation(dsRequest, data);
            }
        }

        
        var inputs = this.getServiceInputs(dsRequest);

        // client only requests: will be handled entirely in-browser, by returning results from
        // an in-browser cache.
        
        var internalCacheRequest = dsRequest.shouldUseCache === true ||
                                    (this.cacheAllData && 
                                    dsRequest.operationType == "fetch" &&
                                    !dsRequest.cachingAllData && 
                                    !dsRequest.downloadResult &&
                                    !(this.cacheAllOperationId &&
                                    this.cacheAcrossOperationIds == false && 
                                    dsRequest.operationId != this.cacheAllOperationId));

        // clientCustom dataProtocol - in this case we assume transformRequest has been implemented
        // to call a server via custom code or get data in some non-network manner (eg from an
        // Applet).  transformRequest was called by getServerInputs() above, so we're done -
        // we're now waiting for the developer's code to call processResponse with a
        // dsResponse.
        
        if (inputs.dataProtocol == "clientCustom" && !internalCacheRequest) return;

        
        // form an rpcRequest for this dsRequest
        var rpcRequest = isc.addProperties({}, dsRequest, inputs);
        // hold onto the dsRequest that initiated this call, and the indirect callback
        rpcRequest._dsRequest = dsRequest;

        // HACK: if we don't do this, when this request is passed to rpcRequest.send(),
        // dsRequest.data gets passed as a requestBody to the proxy servlet
        if (inputs.data == null) rpcRequest.data = null;

        if (this.clientOnly || internalCacheRequest) {
            rpcRequest.clientOnly = true;
            rpcRequest.callback = {target:this, methodName:"_handleClientOnlyReply" };

            isc.RPC.sendRequest(rpcRequest);
            return;
        }

        var opBinding = this.getOperationBinding(dsRequest);
        rpcRequest.transport = opBinding.dataTransport || this.dataTransport;
        if (rpcRequest.transport == "scriptInclude") {
            // cross-site JSON: we will receive live JS objects directly
            rpcRequest.callback = { target:this, methodName:"_handleJSONReply" };
            // transformResponse or previous application via requestProperties in
            // performDSOperation takes precedence
            if (!rpcRequest.callbackParam) {
                rpcRequest.callbackParam = opBinding.callbackParam || this.callbackParam;
            }
            isc.rpc.sendRequest(rpcRequest);
            return;
        }

        var dataFormat = this.getDataFormat(dsRequest);
        // XML pathway: result will be parsed as XML, have recordXPath and schema applied
        if (dataFormat == "xml") {
            var spoofedResponse = rpcRequest.spoofedResponse;
            if (!spoofedResponse) {
                rpcRequest.callback = { target:this, method: this._handleXMLReply };
                isc.xml.getXMLResponse(rpcRequest);
            } else {
                var ds = this;
                isc.Timer.setTimeout(function () {
                    ds._handleXMLReply(isc.xml.parseXML(spoofedResponse), 
                                       spoofedResponse, 
                                       { status:0, httpResponseCode:200, 
                                         data:spoofedResponse },
                                       rpcRequest);
                });
            }

        // JSON pathway: result will be eval()d as JSON, have recordXPath and schema applied
        } else if (dataFormat == "json") {
            rpcRequest.callback = { target:this, method: this._handleJSONTextReply };
            isc.rpc.sendProxied(rpcRequest);

        // CSV pathway: will be converted to JSON and then processed as usual
        } else if (dataFormat == "csv") {
            rpcRequest.callback = { target:this, method: this._handleCSVTextReply };
            isc.rpc.sendProxied(rpcRequest);
    
        // custom pathway: you receive the raw String result and must fill out a DSResponse
        } else {
            rpcRequest.serverOutputAsString = true;
            rpcRequest.callback = { target:this, method:this._handleCustomReply };
            isc.rpc.sendProxied(rpcRequest);
        }
    },
    
    _applySparseAndNoNullUpdates : function (values, oldValues, operationType, paramDS) {
    
        var ds = paramDS || this;
    
        if (!ds.noNullUpdates) {
            if (!ds.sparseUpdates) return;
            if (oldValues == null) return;
        }

        var propertiesToSkip = {
            __ref: true,
            __module: true
        };

        for (var key in values) {

            // Don't touch functions, classes, instances or properties explicitly called out 
            // in the proeprtiesToSkip object above
    		if (isc.isA.Function(values[key])) continue;
            if (propertiesToSkip[key] == true) continue;
            if (isc.isAn.Instance(values[key]) || isc.isA.Class(values[key])) continue;

            var value = values[key];
            if (ds.noNullUpdates && value === null) {
                if (operationType == "add" && ds.omitNullDefaultsOnAdd == true) {
                    delete values[key];
                } else {
                    var field = ds.getField(key),
                        undef;
                    if (field && field.nullReplacementValue !== undef) {
                        values[key] = field.nullReplacementValue;
                    } else {
                        var type = isc.SimpleType.getBaseType(field.type, ds);
                        if (type == "integer") {
                            values[key] = ds.nullIntegerValue;
                        } else if (type == "float") {
                            values[key] = ds.nullFloatValue;
                        } else if (type == "date" || type == "time") {
                            values[key] = ds.nullDateValue;
                        } else if (type == "boolean") {
                            values[key] = ds.nullBooleanValue;
                        } else {   // Anything else, use the string default
                            values[key] = ds.nullStringValue;
                        }
                        
                    }
                }
            // Sparse updates only apply to "update" operations
            } else if (ds.sparseUpdates && operationType == "update") {
                if (oldValues == null) continue;
                var oldValue = oldValues[key];
                if (oldValue == null && !(value == null)) continue;
                if (ds != null) {
                    var field = ds.getField(key);
                    if (field && field.primaryKey) continue;
                }
                if (isc.isA.Date(value) && Date.compareDates(value, oldValue) == 0) {
                    delete values[key];
                } else if (isc.isAn.Array(value)) {
                    for (var i = 0; i < value.length; i++) {
                        this._applySparseAndNoNullUpdates(value[i], oldValue[i], operationType,
                                        field == null ? null : isc.DataSource.get(field.type));
                        var keyCount = 0;
                        for (var k2 in value[i]) keyCount++;
                        if (keyCount == 0) delete value[i];
                    }
                    // If this process leaves an array full of nulls, don't bother serializing it
                    var goodEntry = false;
                    for (var i = 0; i < value.length; i++) {
                        if (value[i] != null) {
                            goodEntry = true;
                            break;
                        }
                    }
                    if (!goodEntry) delete values[key];
                } else if (isc.isAn.Object(value)) {
                    this._applySparseAndNoNullUpdates(value, oldValue, operationType,
                                        field == null ? null : isc.DataSource.get(field.type));
                    // If this process leaves an object with no properties, get rid of it
                    var keyCount = 0;
                    for (var k2 in value) keyCount++;
                    if (keyCount == 0) delete values[key];
                } else if (value == oldValue) {
                    delete values[key];
                }
            }
        }
    },
    
    
    _cloneValues : function (values, dataSource, dupList) {
    
        if (values == null) return;

        var dsDeepClone = dataSource ? dataSource.deepCloneOnEdit : this.deepCloneOnEdit,
            deepClone = dsDeepClone == null ? isc.DataSource.deepCloneOnEdit 
                                              : dsDeepClone;
    
        if (isc.isAn.Array(values)) {
            var storedValues = [];
            for (var i = 0; i < values.length; i++) {
                var value = values[i];
                if (isc.isA.Function(value)) continue;
                if (isc.isAn.Instance(values[prop]) || isc.isA.Class(values[prop])) continue;
                
                if (value == null || isc.isA.String(value) || isc.isA.Boolean(value) ||
                    isc.isA.Number(value))
                {
                    storedValues[storedValues.length] = value;
                
                } else if (isc.isA.Date(value)) {
                    storedValues[storedValues.length] = new Date(value.getTime());

                } else if (isc.isAn.Object(value)) {
                    storedValues[storedValues.length] = this._cloneValues(value, 
                                                                dataSource, dupList);
                }
            }
            return storedValues;
        }
        
        var storedValues = {};
    
        
        if (values._isc_tree != null) {
            values = isc.JSONEncoder._serialize_cleanNode(values)
        }
                
        var propertiesToSkip = {
            __ref: true,
            __module: true
        }; 
                
        if (isc.DataSource.cloneValuesSafely) {
            if (!dupList) dupList = [];
            if (dupList.contains(values)) {
                storedValues = values;
                return;
            }
            dupList.add(values);
        }
    
        for (var prop in values) {
    		if (isc.isA.Function(values[prop])) continue;

            if (propertiesToSkip[prop] == true) continue;
            
            if (isc.isAn.Instance(values[prop]) || isc.isA.Class(values[prop])) continue;
                
            // Special case for dates - duplicate them rather than copying the object 
            // across
            var propValue = values[prop];
            if (isc.isA.Date(propValue)) {
                storedValues[prop] = propValue.duplicate();
                
            } else if (isc.isAn.Object(propValue)) {   // This case works for arrays as well
                var field = dataSource ? dataSource.getField(prop) : null;
                if (!field) {
                    storedValues[prop] = values[prop];
                } else {
                    // Only deep-clone objects corresponding to fields that have deepCloneOnEdit 
                    // set (or inherited from the DataSource or from the static setting on 
                    // isc.DataSource)
                    if (field.deepCloneOnEdit == true || 
                           (field.deepCloneOnEdit == null && deepClone))
                    {
                        // If the global DataSource flag "cloneValuesSafely" is set, keep track of
                        // objects we've seen before and shallow clone any duplicates
                        if (isc.DataSource.cloneValuesSafely) {
                            if (dupList.contains(propValue)) {
                                storedValues[prop] = values[prop];
                                continue;
                            }
                            dupList.add(propValue);
                        }
                        storedValues[prop] = this._cloneValues(propValue,
                            isc.DataSource.get(field.type));
                    } else {
                        storedValues[prop] = values[prop];
                    }
                }
            } else {
                storedValues[prop] = values[prop];
            }

    	}
        return storedValues;
    },
    
    //> @method DataSource.useOfflineResponse()
    // Override point to allow application code to suppress use of a particular offline
    // response.  For example, application code may wish to examine the response's 
    // +link{DSResponse.offlineTimestamp,offlineTimestamp} to make a decision about whether
    // the response is too stale to be useful.
    // <p>
    // This is an application override point only; there is no default implementation.
    //
    // @param dsRequest (DSRequest) The dsRequest object
    // @param dsResponse (DSResponse) The corresponding dsResponse object returned from 
    //                                offline cache
    // @return (boolean) true to allow this response to be used, false to prevent it
    // @group offlineGroup
    // @visibility external
    //<
    
    

    
    fulfilledFromOffline : function (dsRequest) {
        var cachedRequest = dsRequest.unconvertedDSRequest ? 
                            dsRequest.unconvertedDSRequest : dsRequest;
        if (this.useOfflineStorage && isc.Offline) {
            var requestString = dsRequest.dataSource + "." + dsRequest.operationType;
            if (isc.Offline.isOffline()) {
                var dsResponse = isc.Offline.getResponse(cachedRequest);
                this.logInfo("currently offline, for request: " + requestString + 
                             " found cached response: " + this.echoLeaf(dsResponse), 
                             "offline"); 
                if (this.useOfflineResponse && !this.useOfflineResponse(cachedRequest, dsResponse)) {
                    this.logInfo("User-written useOfflineResponse() method returned false; " +
                                 "not using cached response", "offline");
                    dsResponse = null;
                }
                this.processOfflineResponse(dsRequest, dsResponse);
                // whether there's a cached response or not, we're done
                return true;
            } else if (cachedRequest.useOfflineCache || cachedRequest.useOfflineCacheOnly) {
                var dsResponse = isc.Offline.getResponse(cachedRequest);
                if (dsResponse != null) {
                    // found a cached response, return it
                    this.logInfo("request: " + requestString + 
                                 ", returning cached offline response", "offline"); 
                    if (this.useOfflineResponse && !this.useOfflineResponse(cachedRequest, dsResponse)) {
                        this.logInfo("User-written useOfflineResponse() method returned false; " +
                                     "not using cached response", "offline");
                        dsResponse = null;
                    }
                    this.processOfflineResponse(dsRequest, dsResponse);
                    return true;
                } else if (dsRequest.useOfflineCacheOnly) {
                    // no cached response, but we're only allowed to use offline data
                    this.logInfo("request: " + requestString + 
                                 ": useOfflineCacheOnly: no response available", "offline"); 
                    this.processOfflineResponse(dsRequest);
                    return true;
                }
                this.logInfo("request: " + requestString + 
                             ", no cached response, proceeding with network request", "offline"); 
            }
        }
        return false;
    },

    processOfflineResponse : function (dsRequest, dsResponse) {
        // NOTE you get this response if the network is not available but also if you set
        // useOfflineCacheOnly and no cached response is available
        if (!dsResponse) {
            dsResponse = {
                status: isc.RPCResponse.STATUS_OFFLINE,
                data: isc.DataSource.offlineMessage
            };
        }
        
        dsResponse.clientContext = dsRequest.clientContext;
        dsResponse.internalClientContext = dsRequest.internalClientContext;
        dsResponse.isCachedResponse = true;
        // this injects the cached response into the processing chain right at the point
        // where it was previously stored off
        this.fireResponseCallbacks(dsResponse, dsRequest);
    },

    performSCServerOperation : function (dsRequest, data) {
        this.logWarn("Attempt to perform iscServer request requires options SmartClient server " +
                     "support - not present in this build.\nRequest details:"+ 
                     this.echo(dsRequest));
        return;
    },
    
    //> @method dataSource.handleError() (A)
    // If you define this method on a DataSource, it will be called whenever the server returns
    // a DSResponse with a status other than +link{RPCResponse.STATUS_SUCCESS}.  You can use
    // this hook to do DataSource-specific error handling.  Unless you return
    // <code>false</code> from this method, +link{RPCManager.handleError()} will be called by
    // SmartClient right after this method completes.
    // 
    // @param response (DSResponse) the DSResponse or DSResponse object returned from the server
    // @param request (DSRequest) the DSRequest or DSRequest that was sent to the server
    // 
    // @return (boolean) false to suppress +link{RPCManager.handleError()}
    //
    // @group errorHandling
    // @see RPCManager.handleError()
    // @visibility external
    //<

    //> @type DSCallback
    // A +link{type:Callback} to evaluate when a DataSource request completes.
    // <P>
    // The parameters available in the DSCallback expression are:
    // <ul>
    // <li>dsResponse: a +link{class:DSResponse} instance with metadata about the returned data
    // <li>data: data returned to satisfy the DataSource request.  See the
    //     +link{group:dataSourceOperations,DataSource operations} topic for expected results
    //     for each type of DataSource operation
    // <li>dsRequest: the +link{class:DSRequest} that was sent.  You can use
    //     +link{attr:dsRequest.clientContext} to track state during the server turnaround.
    // </ul>
    // For example, if you had a DynamicForm with ID "myForm" and you wanted to retrieve a 
    // record from a DataSource "myUsers", where each record has a "userId" field:
    // <pre>
    //     myUsers.fetchData({ userId : "fred" }, "myForm.setValues(data)");
    // </pre>
    // or
    // <pre>
    //     myUsers.fetchData({ userId : "fred" }, function (dsResponse, data, dsRequest) {
    //                                              myForm.setValues(data);
    //                                            });
    // </pre>
    // <P>
    // Note that if the request encounters a low-level error (such as 500 server error), by
    // default the callback will <b>not</b> be fired, instead, +link{DataSource.handleError()}
    // is called to invoke the default system-wide error handling.  Set
    // +link{rpcRequest.willHandleError,willHandleError}:true to have your callback invoked
    // regardless of whether there are errors, however, make sure your callback properly
    // handles malformed responses when +link{dsResponse.status} is non-zero.
    //
    // @treeLocation Client Reference/Data Binding/DataSource
    // @visibility external
    //<

//> @class DSResponse
// Response sent by the server in response to a +link{DSRequest,DataSource request}.  Contains
// all the properties available on the basic +link{class:RPCResponse}, in addition to the
// properties listed here.
//
// @inheritsFrom RPCResponse
// @treeLocation Client Reference/Data Binding
// @visibility external
//<

//> @attr dsResponse.clientContext (Object : null : R)
//
// The +link{dsRequest.clientContext} object as set on the +link{DSRequest}.
//
// @see dsRequest.clientContext
// @see rpcResponse.clientContext
//
// @visibility external
//<

//> @attr dsResponse.dataSource (String : null : IR)
// The DataSource of this DSResponse.
//
// @group dsResponse
// @visibility external
//<

//> @attr dsResponse.operationType (DSOperationType : null : IR)
// The operation type of the request corresponding to this DSResponse.
//
// @group dsResponse
// @visibility external
//<

//> @attr dsResponse.data (Array of Record : null : IR)
// For "fetch" operations, this is the array of Records fetched.  For "update", "add", and
// "remove" operations, this is typically an array containing a single Record representing
// the record that was updated, added, or removed.
//
// @group dsResponse
// @visibility external
//<

//> @attr dsResponse.status (int : : IR)
//
// Same meaning as +link{RPCResponse.status}, except DSResponses have additional error codes,
// such as +link{RPCResponse.STATUS_VALIDATION_ERROR,validation failure}.
// 
// @see group:dataSourceOperations for standard return types for DataSource Operations
// @group errorHandling
// @visibility external
//<

//> @attr dsResponse.queueStatus (int : : IR)
// An extra property of each DSResponse to a queued request that indicates whether the queue
// as a whole succeeded.  A queueStatus of +link{RPCResponse.STATUS_SUCCESS}, or 0, indicates
// that the queue succeeded whereas a queueStatus of +link{RPCResponse.STATUS_FAILURE}, or -1,
// indicates that the queue failed.
//
// <p>For example, if two "update" requests are sent in a queue and the first succeeded, but
// the second failed validation, then both DSResponses' queueStatus would be -1, but the
// +link{DSResponse.status,status} of the first would be +link{RPCResponse.STATUS_SUCCESS} and
// the status of the second would be an error code such as +link{RPCResponse.STATUS_VALIDATION_ERROR}.
//
// @group errorHandling
// @visibility external
//<

//> @attr dsResponse.errors (Object : null : R)
// Server-side validation errors for an attempted "update" or "add" operation, as a 
// JS Object where each property name is a field name from the record and each property
// value contains error information.
// <P>
// To extract just the simple error strings for each field we recommend passing this
// object to +link{dataSource.getSimpleErrors()}
// <P>
// The Java API DSResponse.addError(fieldName, errorMessage) is used to send server-side
// errors to the client.  See the Java Server Reference for details.
//
// @see dataSource.handleError()
// @group errorHandling
// @visibility external
//<

//> @attr dsResponse.startRow (number : null : R)
// Starting row of returned server results, when using paged result fetching
// <p>
// Note that startRow and endRow are zero-based - the first record is row zero.
//
// @group paging
// @visibility external
//<

//> @attr dsResponse.endRow (number : null : R)
// End row of returned server results, when using paged result fetching
// <p>
// Note that startRow and endRow are zero-based - the first record is row zero.
//
// @group paging
// @visibility external
//<

//> @attr dsResponse.totalRows (number : null : R)
// Total number of rows available from the server that match the current filter criteria,
// when using paged result fetching.
//
// @group paging
// @visibility external
//<

//> @attr dsResponse.invalidateCache (boolean : null : R)
// Optional flag that can be set by the server to force ResultSets to drop any caches of
// records from the DataSource that was the target of the operation.
//
// @group cacheSync
// @visibility external
//<

//> @attr dsResponse.cacheTimestamp (number : null : R)
// Optional number sent by the server to allow cache staleness detection.
// <P>
// If sent by the server, the cacheTimestamp will be sent along with any fetch DSRequests
// originated by ResultSets who received a DSResponse with a
// <code>cacheTimestamp</code> set.
// 
// @group cacheSync
//<

//> @attr dsResponse.offlineTimestamp (number : null : R)
// Timestamp (millisecond value) to indicate when this dsResponse was cached in 
// +link{class:Offline,offline storage}.  Not applicable if the response has never been 
// stored offline.
// @group offlineGroup
// @visibility external
//<

//> @attr dsResponse.fromOfflineCache (boolean : null : R)
// If set, indicates that this response came from the offline cache, not the server.  This 
// flag is the only reliable way for application code to determine the source of a response.
// @group offlineGroup
// @visibility external
//<

//> @attr dsResponse.httpHeaders (Object : null : R)
// HTTP headers returned by the server as a map from header name to header value.
// <P>
// Headers are available only when the default +link{RPCTransport} "xmlHttpRequest" is in use,
// and browsers may limit access to headers for cross-domain requests or in other
// security-sensitive scenarios.
//
// @visibility external
//<

// --------------------------------------------------------------------------------------------

//> @class DSRequest
// Request sent to the server to initiate a 
// +link{group:dataSourceOperations,DataSource operation}.  All properties which are legal on
// +link{class:RPCRequest} are legal, in addition to the properties listed here.
// <P>
// 
// 
//
// @inheritsFrom RPCRequest 
// @treeLocation Client Reference/Data Binding
// @see RPCRequest
// @visibility external
//<

//> @attr dsRequest.dataSource (String : null : IR)
// DataSource this DSRequest will act on.
// <P>
// This property is generally automatically populated, for example when calling
// +link{method:DataSource.fetchData()} the dataSource property is set to the target
// DataSource.
// @visibility external
//<

//> @attr dsRequest.operationType (DSOperationType : null : IR)
// Type of operation being performed: "fetch", "add", "remove", "update" or "custom".
// <P>
// This property is generally automatically populated, for example when calling
// <code>fetchData()</code> on a DataSource or DataBound component the operationType is
// automatically set to "fetch".  Note that "custom" operations are never generated
// automatically, they are always fired by your code.
// @visibility external
//<

//> @attr dsRequest.data  (Object or Array of Object : null : IR)
// Data, such as search criteria or an updated record, to be acted upon.  Contents differ by
// <code>operationType</code>, see +link{group:dataSourceOperations,DataSource Operations} for
// details.
// <P>
// This field is generally filled in by passing the "data" argument to methods such as
// +link{method:ListGrid.fetchData()}.
//
// @visibility external
//<

//> @attr dsRequest.callback (DSCallback : null : IR)
// 
// A callback method that will be called with an instance of DSResponse, as sent by the server.
// Queuing does not affect callbacks in any way - your specified callback will be invoked for 
// each DSRequest that contained a callback regardless of whether the request was sent as part
// of a queue or not.
// <P>
// Note that if the request encounters an error (such as 500 server error), by default the
// callback will <b>not</b> be fired.  Instead, +link{RPCManager.handleError()} is called to
// invoke the default system-wide error handling.  Set +link{willHandleError}:true to have your
// callback invoked regardless of whether there are errors; however, make sure your callback
// properly handles malformed responses when +link{RPCResponse.status} is non-zero.  See the
// +link{group:errorHandling,error handling overview} below for more details.
//
// @group errorHandling
// @visibility external
//<

//> @attr dsRequest.startRow (number : null : IR)
// Starting row of requested results, used only with fetch operations.  If unset, 0 is assumed.
// <p>
// Note that startRow and endRow are zero-based, inclusive at the beginning and exclusive at
// the end (like substring), so startRow: 0, endRow: 1 is a request for the first record.
//
// @group paging
// @visibility external
//<

//> @attr dsRequest.endRow (number : null  : IR)
// End row of requested results, used only with fetch operations.
// <p>
// Note that startRow and endRow are zero-based, inclusive at the beginning and exclusive at
// the end (like substring), so startRow: 0, endRow: 1 is a request for the first record.
//
// @group paging
// @visibility external
//<

//> @attr dsRequest.sortBy (String | Array of String or Array of SortSpecifiers : null : IR)
// Fieldname to sortBy, prefixed with optional "-" indicating descending sort.  For example, to
// sort by the field "userName" in ascending order, set <code>sortBy</code> to just
// "userName".  For descending sort on "userName", set <code>sortBy</code> to "-userName".  
// <P>
// To sort by multiple fields, an Array of Strings is also supported.  For example, to sort by
// the field "department" in ascending order, followed by the field "userName" in descending
// order, set <code>sortBy</code> to: 
// <P>
// <code>[ "department", "-userName" ]</code>
// <P>
// Additionally, this property supports an array of +link{SortSpecifier} objects.  Setting 
// <code>sortBy</code> to the following SortSpecifier array results in the same multi-level
// sort mentioned above:
// <P>
// <code>
// [<br>
// &nbsp;&nbsp;{ property: "department", direction: "ascending" },<br>
// &nbsp;&nbsp;{ property: "userName", direction: "descending" }<br>
// ]
// </code>
//
// @visibility external
//<

//> @attr dsRequest.groupBy (Array of String : null : IR)
// For <code>fetch</code> operation group by fields can be set. In this case only fields that appear in this
// <code>Array</code> and in <code>dsRequest.fieldFunctions</code> attribute will be fetched.<br>
// Holds <code>Array</code> with field names to fetch.
//
// @see attr:dsRequest.fieldFunctions
// @visibility serverGrouping
//<

//> @attr dsRequest.fieldFunctions (Object : null : IR)
// For <code>fetch</code> operation aggregation functions can be set. In this case only fields that appear in this
// <code>Map</code> and in <code>dsRequest.groupBy</code> attribute will be fetched.<br>
// Holds <code>Map</code> with field names as keys and aggregation functions as values.
//
// @see attr:dsRequest.groupBy
// @visibility serverGrouping
//<

//> @attr dsRequest.outputs (String : null : IR)
// The list of fields to return in the response, specified as a comma-separated string (eg,
// <code>"foo, bar, baz"</code>).  You can use this property to indicate to the server that 
// you are only interested in a subset of the fields that would normally be returned.
// <p>
// Note that you cannot use this property to request a <em>superset</em> of the fields that
// would normally be returned, because that would be a security hole.  It is possible to 
// configure individual +link{class:operationBinding}s to return extra fields, but this must
// be done in the server's +link{class:DataSource} descriptor; it cannot be altered on the fly
// from the client side.
//
// @see attr:operationBinding.outputs
// @see attr:dsRequest.additionalOutputs
// @visibility external
//<

//> @attr dsRequest.additionalOutputs (String : null : IRA)
// For fetch, add or update operation, an optional comma separated list of fields to fetch from another,
// related DataSource.
// <P>
// Fields should be specified in the format 
// <code>"localFieldName!relatedDataSourceID.relatedDataSourceFieldName"</code>.
// where <code><i>relatedDataSourceID</i></code> is the ID of the related dataSource, and
// <code><i>relatedDataSourceFieldName</i></code> is the field for which you want to
// fetch related values. The returned field values will be stored on 
// the data returned to the client under the specified <code><i>localFieldName</i></code>.
// Note that this will be applied in addition to any specified +link{dsRequest.outputs}.
// <P>
// Note that as with +link{dataSourceField.includeFrom}, the related dataSource must be
// linked to the primary datasource via a foreignKey relationship.
// <P>
// Note additionalOutputs sent in request from the browser can be completely disabled in 
// +link{server_properties,server.properties} by setting
// <code>datasource.allowClientAdditionalOutputs</code>:
// <pre>
//     datasource.allowClientAdditionalOutputs: false
// </pre>
// In this case +link{dsRequest.additionalOutputs} sent from the browser will be cleared before 
// executing request. Note that programatically configured additionalOutputs are always allowed, but 
// you can't modify them from within a DMI method, so the only way to execute a request 
// with additionalOutputs that differ from what was sent by the client is to create a new DSRequest
//
// @visibility crossDS
//<

//> @attr dsRequest.generateRelatedUpdates (boolean : null : IRW)
// Specifies should related updates have to be generated.
// If not set (or set to <code>null</code>) then related updates will be generated only for
// "add" and "update" operations. This property has to be explicitly set to <code>true</code>
// to generate related updates for "remove" operation.<p/>
// This functionality loads related objects from database thus affecting operation performance.
// For "add" and "update" operations related objects are loaded anyway and performance impact is minimal.
// Simple "remove" operation does not need to load related objects. Depending on database structure
// performance impact can be significant if this property is set to <code>true</code>.
//
// @visibility external
//<

//> @type TextMatchStyle
// For "fetch" operations, how search criteria should be interpreted for text fields.
// @value "exact" test for exact match
// @value "substring" test for case-insenstive substring match
// @value "startsWith" test for the beginning of the value matching the search criteria
//
// @visibility external
//<

//> @attr dsRequest.textMatchStyle (TextMatchStyle: "exact" : IR)
// For "fetch" operations, how search criteria should be interpreted for text fields: either
// "exact" for exact match, "startsWith" for matching at the beginning only, or "substring" for
// case-insensitive substring match.
// <p>
// This setting is respected by the built-in SQLDataSource.  Your custom DataSource
// implementation can interpret the search criteria passed into "fetch" operations in arbitrary
// ways; you can safely ignore this flag and use others of your own devising.
//
// @visibility external
//<

//> @attr dsRequest.oldValues (Record : null : IR)
// For an <code>update</code> or <code>remove</code> operation, the original values from the
// record that is being updated or removed.  <code>oldValues</code> is automatically added to
// DSRequests submitted by DataBound Components.  Available on the server via
// <code>DSRequest.getOldValues()</code>.
// <P>
// The server can compare the <code>oldValues</code> to the most recent stored values in order
// to detect that the user was looking at stale values when the user submitted changes (NOTE:
// this means of detecting concurrent edit is sometimes called "optimistic concurrency" or
// "long transactions").  
// <P>
// In applications where a policy of "last update wins" is not appropriate when updating
// certain fields, special UI can be shown for this case.  For example, on detecting concurrent
// edit, the server may send back a special <code>dsResponse.status</code> code that the client
// application detects, offering the user a choice of proceeding with the operation,
// discarding edits, or reconciling new and old values in a special interface.
//
// @visibility external
//<

//> @attr dsRequest.componentId (String : null : IR)
// For requests submitted by a +link{DataBoundComponent}, the +link{canvas.ID} of the
// submitting component.
// <P>
// This ID will be present for operations including automatic saves by a ListGrid 
// +link{group:editing,during editing}, or calls to
// +link{dynamicForm.saveData,form.saveData()}.  It will not be present for a direct
// call to a DataSource method such as +link{dataSource.fetchData()}.
// <P>
// Note this is the component's <b>String</b> ID - you can retrieve the component itself
// via +link{Canvas.getById()}.
// <P>
// This property should be used for debugging purposes only - do not use it to trigger
// differences in server-side behavior, instead, use +link{dsRequest.operationId} because only
// <code>operationId</code> is considered when assessing
// +link{group:dsRequestEquivalence,request equivalence}.
//
// @visibility external
//<

//> @attr dsRequest.operationId (String : null : IR)
// When a +link{interface:DataBoundComponent} sends a DSRequest, the
// <code>dsRequest.operationId</code> will be automatically picked up from the
// <code>fetchOperation</code>, <code>addOperation</code>, etc properties of the
// DataBoundComponent.  
// <P>
// The <code>operationId</code> serves as an identifier that you can use to create
// variations on the 4 basic DataSource operations that are used by different components in
// different parts of your application.  For example, you may be using a standard
// <code>fetch</code> operation in one part of your application, however on another screen you
// want to perform a <code>fetch</code> operation on the same DataSource but interpret search
// criteria differently (eg full text search).  
// <P>
// If you declare more than one +link{OperationBinding} for the same
// +link{operationBinding.operationType}, you can specify an <code>operationId</code>
// +link{operationBinding.operationId,on the operationBinding} which will cause that
// operationBinding to be used for dsRequests containing a matching <code>operationId</code>.
// This allows all the possible settings of an <code>operationBinding</code>, including
// +link{operationBinding.wsOperation,wsOperation} or +link{dmiOverview,DMI} settings, to be switched on a
// per-component or per-request basis.  
// <P>
// For example, by setting the <code>fetchOperation</code> on a particular ListGrid, you could
// cause it to invoke a different server method via DMI, different
// +link{operationBinding.dataURL,dataURL} or different 
// +link{operationBinding.wsOperation,web service operation}.
// <P>
// The <code>operationId</code> can also be directly received by the server in order to affect
// behavior.  When using the SmartClient Server, <code>operationId</code> can be accessed via
// dsRequest.getOperationId().  The +link{RestDataSource} will also send the
// <code>operationId</code> to the server as part of the
// +link{restDataSource.metaDataPrefix,request metadata}.  
// <P>
// Note that if you +link{dataSource.fetchData,manually invoke} a DataSource operation, you can
// also specify operationId via the <code>requestProperties</code> parameter.
// <P>
// Note that the <code>operationId</code> has special significance in terms of whether two
// DSRequests are considered equivalent for caching and synchronization purposes - see
// +link{group:dsRequestEquivalence}.
//
// @group operations
// @visibility external
//<


//> @groupDef dsRequestEquivalence
// Various subsystems have a need to compare DataSource requests and understand if
// they are equivalent or affect the same data (examples include 
// +link{ResultSet,automatic cache synchronization} and 
// +link{DataSource.useOfflineCache,offline caching and synchronization}).  
// <P>
// Aside from basic properties that would clearly make two DSRequests non-equivalent
// (dataSource, operationType and data, as well as sortBy, startRow, endRow and textMatchStyle
// for a "fetch"), +link{dsRequest.operationId} is the only property that will cause two
// DSRequests to be considered distinct (non-equivalent) requests. 
// <P>
// Bearing this in mind, the best practice is:
// <ul>
// <li> everything that will be treated as criteria or as values on the server side should be
// part of +link{dsRequest.data}.  Do not "smuggle" data that will ultimately be used as
// criteria or values in other dsRequest properties, such as 
// +link{rpcRequest.params,HTTP parameters}.
// <li> use +link{dsRequest.operationId} as the sole piece of information in the request that
// modifies how the request as a whole is executed.  If two or more pieces of information are
// required, combine or encode them into a single operationId String.  If this becomes awkward
// because there are many operation variants, consider including additional fields in
// +link{dsRequest.data} instead.
// </ul>
//
// @visibility external
//<

//> @attr dsRequest.requestId (String : varies : RA)
// Automatically generated unique ID for this request. This ID will be required by developers 
// making use of the +link{type:DSProtocol,"clientCustom" dataProtocol}.
// @visibility external
//<

//> @attr dsRequest.clientContext (Object : {} : IRW)
// An object to be held onto for the duration of the DSRequest turnaround to track
// application-specific context.
// <P>
// When a DataSource request completes, the <code>clientContext</code> is available in the
// +link{type:DSCallback} as <code>dsResponse.clientContext</code>.  The
// <code>clientContext</code> is never sent to the server.
// <br>
// The <code>clientContext</code> is useful for holding onto state that will be used when the
// +link{type:DSCallback} fires, such as the name of a component that will receive the
// returned data.
//
// @see DSResponse.clientContext
// @see RPCRequest.clientContext
//
// @visibility external
//<

//> @attr dsRequest.parentNode (TreeNode : null : R)
// For advanced use in integrating trees that 
// +link{TreeGrid.loadDataOnDemand,load data on demand} with web services,
// <code>parentNode</code> is automatically set in "fetch" DSRequests issued by a
// databound TreeGrid that is loading children for that <code>parentNode</code>.
// <P>
// This is sometimes needed if a web service requires that additional properties beyond the
// ID of the parentNode must be passed in order to accomplished level-by-level loading.
// A custom implementation of +link{dataSource.transformRequest()} can access
// dsRequest.parentNode and add any such properties to +link{dsRequest.data}.
// <P>
// <code>parentNode</code> will also be automatically set by a TreeGrid performing databound
// reparenting of nodes, as implemented by +link{treeGrid.folderDrop}.
// <P>
// This property can only be read.  There is no meaning to setting this property
// yourself.
//
// @visibility external
//<

//> @attr dsRequest.resultTree (ResultTree : null : R)
// For advanced use in integrating trees that 
// +link{resultTree.loadDataOnDemand,load data on demand} with web services,
// the ResultTree that issued this "fetch" DSRequest is automatically made available as the
// <code>resultTree</code> property.
// <P>
// This property can only be read.  There is no meaning to setting this property
// yourself.
//
// @visibility external
//<

//> @attr dsRequest.resultSet (ResultSet : null : R)
// For advanced use in integrating dataset paging with web services,
// the ResultSet that issued this "fetch" DSRequest is automatically made available as the
// <code>resultSet</code> property.
// <P>
// This property can only be read.  There is no meaning to setting this property
// yourself.
//
// @visibility external
//<

//> @attr dsRequest.shouldUseCache (boolean : null : IRW)
// This is a per-request flag for explicitly controlling whether the cache is used (bypassing
// it when not wanted, or using it when settings would indicate otherwise). See 
// +link{dataSource.cacheAllData}, +link{dataSource.cacheAllOperationId} and
// +link{dataSource.cacheAcrossOperationIds} for caching management for all requests of a 
// dataSource.
//
// @visibility external
//<


//> @attr dsRequest.useFlatFields (Boolean : null : IR)
// When <code>useFlatFields</code> is set for a request to be sent to a WSDL web service, when
// creating the input XML message to send to the web service, properties in
// +link{data,request.data} will be used as the values for XML elements of the same name, at
// any level of nesting.
// <P>
// <code>useFlatFields</code> allows you to ignore gratuitous XML message structure, such as
// extra levels of nested elements, and provides some insulation against changes in the
// required structure of the input message.
// <P>
// For example, given this input message:
// <pre>
// &lt;FindServices&gt;
//     &lt;searchFor&gt;search text&lt;/searchFor&gt;
//     &lt;Options&gt;
//         &lt;caseSensitive&gt;false&lt;/caseSensitive&gt;
//     &lt;/Options&gt;
//     &lt;IncludeInSearch&gt;
//         &lt;serviceName&gt;true&lt;/serviceName&gt;
//         &lt;documentation&gt;true&lt;/documentation&gt;
//         &lt;keywords&gt;true&lt;/keywords&gt;
//     &lt;/IncludeInSearch&gt;
// &lt;/FindServices&gt;
// </pre>
// If <code>useFlatFields</code> were <b>not</b> set, in order to fill out this message
// correctly, <code>request.data</code> would need to be:
// <pre>{
//    searchFor: "search text",
//    Options : {
//        caseSensitive: false,
//    },
//    IncludeInSearch : {
//        serviceName: true,
//        documentation : true,
//        keywords : true
//    }
// }</pre>
// However if useFlatFields were set, <code>request.data</code> could be just:
// <pre>{
//    searchFor: "search text",
//    caseSensitive: false,
//    serviceName: true,
//    documentation : true,
//    keywords : true
// }</pre>
// <code>useFlatFields</code> is often set when the input data comes from a +link{DynamicForm}
// to avoid the cumbersome and fragile process of mapping input fields to an XML structure.
// <P>
// +link{operationBinding.useFlatFields} can also be set to cause <b>all</b> dsRequests of a
// particular type to <code>useFlatFields</code> automatically.
// <P>
// For +link{DataBoundComponent,DataBoundComponents}, 
// +link{DataBoundComponent.useFlatFields,component.useFlatFields} can be set use "flattened"
// binding to fields of a WSDL message or XML Schema.
// <P>
// Note that <code>useFlatFields</code> is not generally recommended for use with XML input
// messages where multiple simple type fields exist with the same name, however if used in this
// way, the first field to use a given name wins.  "first" means the first field encountered in a
// depth first search.  "wins" means only the first field will be populated in the generated
// XML message.
//
// @group flatFields
// @visibility xmlBinding
//<

//> @attr dsRequest.useFlatHeaderFields (boolean : null : IRW)
// Cause the +link{dsRequest.useFlatFields,useFlatFields} XML serialization behavior to be used
// for <b>all</b> soap headers in the request.  See also +link{dsRequest.headerData}.
//
// @group flatFields
// @visibility xmlBinding
//<

//> @attr dsRequest.headerData (Object : null : IRW)
// For DataSources using SOAP messaging with a WSDL web service, data to be serialized to form
// SOAP headers, as a map from the header part name to the data.  See
// +link{WSRequest.headerData} for more information.
// <P>
// SOAP headers typically contain request metadata such as a session id for authentication, and
// so <code>dsRequest.headerData</code> is typically populated by
// +link{dataSource.transformRequest()}, or, for data that applies to every request sent to the
// server, by +link{webService.getHeaderData()}.
// <P>
// Note that this only applies to SOAP headers. General HTTP headers for requests may be
// modified using +link{RPCRequest.httpHeaders}.
//
// @visibility xmlBinding
//<

//> @type ExportFormat
// One of the supported formats for data-export.  If you are doing a 
// +link{DataBoundComponent.exportClientData(),client export} to one of the native spreadsheet
// formats (xls or ooxml), we also export +link{object:Hilite,hilite-based} coloring.  So, if
// Hilites are causing a particular cell to be rendered as green text on a blue background, 
// the corresponding cell in the exported spreadsheet document will also be colored that way.
//
// @value "xml"    Export data as XML records
// @value "json"   Export data as JSON objects
// @value "csv"    Export data in comma-separated format
// @value "xls"    Export data in native Microsoft Excel 97 format
// @value "ooxml"  Export data in native Microsoft Excel 2007 format (also called XLSX)
//
// @serverDS allowed
// @visibility external
//<

//> @type ExportImageFormat
// One of the supported formats for image export.
//
// @value "png"    Export as PNG
// @value "jpeg"   Export as JPEG
//
// @serverDS allowed
// @visibility external
//<

//> @type ExportDisplay
// Method to use for displaying the exported data.
//
// @value "download"    Show the Save As dialog and download the file
// @value "window"      Show the data in a new browser window
// @value "return"      Return the data for further programmatic processing in the browser
//
// @serverDS allowed
// @visibility external
//<

//> @type LineBreakStyle
// The style of line-breaks to use when exporting data
//
// @value "default"    Use the default line-break style of the server OS
// @value "unix"       Use UNIX-style line-breaks (LF only)
// @value "mac"        Use MAC-style line-breaks (CR only)
// @value "dos"        Use DOS-style line-breaks (both CR & LF)
//
// @serverDS allowed
// @visibility external
//<

//> @attr dsRequest.exportResults (Boolean : false : IR)
// When set, causes the results of the DSRequest to be exported to a file, whose name 
// and format are indicated by +link{dsRequest.exportFilename} and 
// +link{dsRequest.exportAs} respectively.  When no exportFilename is provided, the default is
// <i>Results</i> and the default value of exportAs is <i>csv</i>.  
// <P>
// The export field-list can also be configured, see +link{dsRequest.exportFields}.
// <P>
// Once the operation completes, +link{dsRequest.exportDisplay} specifies whether the exported
// data should be downloaded to the file-system or displayed in a new window.  The default value
// of exportDisplay is "download" which displays the Save As dialog.  See +link{ExportDisplay} 
// for more information.
// <P>
// You can configure the style of +link{LineBreakStyle, line-breaks} to use when 
// generating the output, the +link{dsRequest.exportDelimiter, delimiter} to use when exporting
// to CSV and the +link{dsRequest.exportTitleSeparatorChar, separator-character} to use in 
// field-titles when exporting to XML.
// <P>
// Additionally, you can output arbitrary text before and after the exported data by setting 
// +link{dsRequest.exportHeader, exportHeader} and +link{dsRequest.exportFooter, exportFooter}.
// <P>
// Note that an export initiated using dsRequest properties does not provide support for JSON
// format (see 
// <a href="http://forums.smartclient.com/showthread.php?t=235">this post</a> for more detail).
// <P>
// As well as setting dsRequest.exportResults and related properties, exports can be initiated
// in two other ways, via +link{OperationBinding}s and via custom server code which sets 
// export-related properties on the +link{DSResponse}.  Both of those methods support exporting
// to JSON format.
// <P>
// <b>Format Examples</b>
// XML format
// <pre>
//     &lt;List&gt;
//         &lt;Object&gt;
//             &lt;id&gt;10101&lt;/id&gt;
//             &lt;displayName&gt;Record 10101&lt;/displayName&gt;
//         &lt;/Object&gt;
//    &lt;/List&gt;
// </pre>
// JSON Format
// <pre>
//     [
//         { id: 10101, displayName: "Record 10101" }
//     ]
// </pre>
// CSV Format
// <pre>
//     id,displayName
//     10101,"Record 10101"
// </pre>
//
// @visibility external
//<

//> @attr dsRequest.exportAs (ExportFormat : "csv" : IR)
// The format in which the data should be exported.  See +link{ExportFormat} for more 
// information.
//
// @visibility external
//<

//> @attr dsRequest.exportFilename (String : null : IR)
// The name of the file to save the exported data into.  If 
// +link{dsRequest.exportToFilesystem,exportToFilesystem} is set, this is the name of the 
// file the server creates on its filesystem.  If +link{dsRequest.exportToClient,exportToClient}
// is set, this is the filename that will appear to the browser.
//
// @see dsRequest.exportPath
// @visibility external
//<

//> @attr dsRequest.exportImageFormat (ExportImageFormat : "png" : IR)
// The image format in which the SVG graphic should be exported.
//
// @visibility external
//<

//> @attr dsRequest.exportImageQuality (float : null : IR)
// If exporting in +link{ExportImageFormat,JPEG format}, the output JPEG quality level.
// This is a number from 0 to 1, with 1 representing the best quality and 0 representing the
// least quality but smallest file size.
//
// @visibility external
//<

//> @attr dsRequest.exportPath (String : null : IR)
// If +link{dsRequest.exportToFilesystem,exportToFilesystem} is set, optionally specifies a 
// path to use when saving the file.  This path is relative to the default export path, which
// is set using the +link{server_properties,server.properties} setting <code>export.location</code>; this
// is the project webRoot by default.  For example, with the default setting of 
// <code>export.location</code>, an <code>exportPath</code> of <code>"shared/ds"</code> and 
// an +link{dsRequest.exportFilename,exportFilename} of <code>"exportedData.csv"</code>,
// SmartClient Server would export to file <code>$webRoot/shared/ds/exportedData.csv</code>.
// <p>
// If you do not specify this property, SmartClient Server will export to the file indicated 
// by <code>exportFilename</code> directly in the default export location.
// <p>
// This property is only applicable when +link{dsRequest.exportToFilesystem,exportToFilesystem}
// is set.
//
// @see dsRequest.exportFilename
// @visibility external
//<

//> @attr dsRequest.exportToFilesystem (Boolean : false : IR)
// If set, SmartClient Server will export data to a file on the <b>server</b> filesystem.  The
// file we export to is determined by the +link{dsRequest.exportFilename,exportFilename} and 
// +link{dsRequest.exportPath,exportPath}.  Note that filesystem exports are disabled by 
// default, for security reasons.  To enable them, set <code>export.allow.filesystem</code> to
// true in your <code>server.properties</code> file.  If you enable filesystem exports, you 
// should also consider setting a default export path, as described in the 
// +link{exportPath,exportPath} documentation.
// <p>
// Note that it is perfectly valid to specify both this property and 
// +link{dsRequest.exportToClient,exportToClient}; in this case the data is both exported to a
// file on the server filesystem <em>and</em> downloaded to the client.  If you specify 
// <em>neither</em> property, the export no-ops.
// <p>
// It is possible to redirect the filesystem export to make use of an <code>OutputStream</code>
// you provide.  You use this when you want to make some use of the export document other than 
// writing it to a disk file - for example, attaching it to an email or writing it to a 
// database table.  See the server-side Javadocs for <code>DSRequest.setExportTo()</code>.
// @visibility external
//<

//> @attr dsRequest.exportToClient (Boolean : true : IR)
// If set, SmartClient Server will export data back to the client, either as a file download 
// or as content in a new browser window, depending on the setting of
// +link{dsRequest.exportDisplay,exportDisplay}.
// <p>
// Note that it is perfectly valid to specify both this property and 
// +link{dsRequest.exportToFilesystem,exportToFilesystem}; in this case the data is both 
// exported to a file on the server filesystem, and downloaded to the client.  If you specify 
// <em>neither</em> property, the export no-ops.
//
// @visibility external
//<

//> @attr dsRequest.exportTitleSeparatorChar (String : null : IR)
// The character with which to replace spaces in field-titles when exporting to XML.
//
// @visibility external
//<

//> @attr dsRequest.exportDisplay (ExportDisplay : "download" : IR)
// Specifies whether the exported data will be downloaded as an attachment or displayed in a 
// new browser window. See +link{ExportDisplay} for more information.
//
// @visibility external
//<

//> @attr dsRequest.lineBreakStyle (String : null : IR)
// The style of line-breaks to use in the exported output.  See +link{LineBreakStyle} for
// more information.
//
// @visibility external
//<

//> @attr dsRequest.exportDelimiter (String : "," : IR)
// The character to use as a field-separator in CSV exports.  The default delimiter is comma.
//
// @visibility external
//<

//> @attr dsRequest.exportShowHeaderSpanTitles (Boolean : true : IR)
// When you erxport a +link{class:ListGrid} that has +link{ListGrid.headerSpans,headerSpans},
// should headerSpans also be exported.  See +link{exportSpanTitleSeparator} for details of
// of what it means to export headerSpans to different export targets.
//
// @visibility external
//<

//> @attr dsRequest.exportSpanTitleSeparator (String : " - " : IR)
// When you export a +link{class:ListGrid} that has +link{ListGrid.headerSpans,headerSpans}
// defined and +link{dsRequest.exportShowHeaderSpanTitles} is true, the behavior is dependent
// on the export type.  Direct exports to Excel formats (both XLS and OOXML) place the 
// headerSpans in merged cells in the spreadsheet, giving the same visual effect as the 
// original ListGrid.  This is not possible with exports to CSV format; instead, we alter the 
// exported headers so that they contain the titles of the ancestor headerSpan(s).
// <p>
// For example, if you had a field titled "Population" inside a headerSpan titled "National",
// nested inside another headerSpan titled "Demographics", that would result in the exported
// field being titled "Demographics - National - Population".
// <p>
// The <code>exportSpanTitleSeparator</code> property allows you to override the separator 
// string used when constructing these amalgamated headers.
//
// @visibility external
//<

//> @attr dsRequest.exportFields (Array : null : IR)
// The list of field-names to export.  If provided, the field-list in the exported output is 
// limited and sorted as per the list.
// <P>
// If exportFields is not provided, the exported output includes all visible fields 
// from the DataSource (field.hidden=false), sorted in the order they're defined.
//
// @visibility external
//<

//> @attr dsRequest.exportHeader (String : null : IR)
// Optional text to appear at the beginning of the file.
//
// @visibility external
//<

//> @attr dsRequest.exportHeaderless (Boolean : false : IRW)
// This property allows omitting column names from CSV and Excel exports (no effect on JSON or XML exports).
//
// @visibility external
//<

//> @attr dsRequest.exportFooter (String : null : IR)
// Optional text to appear at the end of the file.
//
// @visibility external
//<

//> @attr dsRequest.exportData (Array of Record : null : IR)
// Only applies to request properties passed to +link{dataBoundComponent.exportClientData()}.
// If specified this property contains an arbitrary set of data to be exported.
// @visibility external
//<

//> @attr dsRequest.exportValueFields (boolean : null : IR)
// Only applies to request properties passed to +link{dataBoundComponent.exportClientData()}.
// Ordinarily, any fields that have a +link{ListGridField.displayField,displayField} defined 
// have the value of that displayFIeld exported, rather than the underlying value in the 
// +link{ListGridField.valueField,valueField}.  If you set this  property, we export both 
// the underlying value and the displayField value.
// @visibility external
//<

//> @attr dsRequest.exportDatesAsFormattedString (boolean : null : IR)
// Only applicable when exporting to native spreadsheet formats <code>XLS</code> and 
// <code>OOXML</code>.
// <p>
// If set, we export date fields as strings that exactly match the formatting present in the 
// +link{class:DataBoundComponent} from which we are exporting.  Note that this will mean 
// the values are plain strings in the spreadsheet.
// <p>
// The default behavior when this property is not set is to export date fields as real date 
// values in the spreadsheet, using whichever date format is prevalent (the format specified on
// the field in the component, the dataSourceField or the current system default, in that 
// order).  This may result in the spreadsheet cell having a different format to the value 
// that the user sees in the <code>DataBoundComponent</code> we are exporting from.
//
// @visibility external
//<

//> @type ValidationMode
// Mode of validation used for running validation on partially entered data.
//
// @value "full"    Assume provided data is complete and run all validation rules
// @value "partial" Validate only the data values that are present ignoring the
//                  absence of required fields, validators that refer to fields
//                  that are not present, and any conditional validators dependent
//                  on fields that are not present.
//
// @serverDS allowed
// @visibility external
//<

//> @attr dsRequest.validationMode (ValidationMode : "full" : IR)
// Mode of validation for entered data.
//
// @visibility external
//<

//> @attr dsRequest.pendingAdd (boolean : null : IR)
// Indicates that a validation request is being made for a record that will ultimately be 
// saved with an "add" request, as opposed to an "update" request.  This context is necessary
// for some validators because the nature of the validation depends on whether we are adding
// or updating a record.  The system sets this flag when processing interim validations, such
// as those fired when +link{dynamicForm.validateOnChange} is in force.
//
// @visibility external
//<

//> @attr dsRequest.parameters (Object : null : IR)
// Parameters to pass to the server <i>in addition to</i> the standard inbound data required by
// a DataSource operation of this type.
// <p>
// Parameters are used only with custom implementations of DataSource operations; the
// SQLDataSource built-in to the ISC server ignores all parameters.  A typical use of
// parameters is to control an extra action that a custom DataSource operation supports.  For
// example, a custom "update" DataSource operation might support sending an email notification
// whenever a record is updated; parameters could control whether this notification email is
// sent, to what list of people, etc.
// <p>
// An alternative to using parameters is to use +link{method:RPCManager.startQueue()} to
// combine multiple distinct operations into one round-trip.
// <p>
// These parameters are available on the server via <code>dsRequest.getParameter()</code>.
//<



//> @attr dsRequest.streamResults (String : false : IR)
// If true, results will be streamed on the server, rather than all records being read into 
// server memory at once; this approach is appropriate for retrieving or exporting large
// datasets without swamping the server.  
// <p>
// Although this property can be set without any particular concerns (small datasets can be 
// streamed just as readily as large ones), bear in mind that although streaming enables the
// processing of very large datasets, processing and downloading very large datasets in a 
// normal client/server flow will very rarely give an acceptable user experience.  Streaming
// is of more practical use in a batch setting - for example, a disconnected 
// +link{dsRequest.exportToFilesystem,export}.
// <p>
// Note that streaming requires specific server support; of SmartClient's built-in DataSource
// types, only <code>SQLDataSource</code> is able to stream results.  This property is ignored
// by other DataSource types.
// <p>
// See also the server-side documentation for <code>DSResponse</code>, 
// <code>SQLDataSource</code> and <code>StreamingResponseIterator</code>.
//
// @visibility external
//<

	
//> @attr dsRequest.progressiveLoading (boolean : null : IRW)
// Sets +link{DataSource.progressiveLoading,progressive loading mode} for this specific
// request, overriding the OperationBinding- and DataSource-level settings.  Note that this 
// setting applies only to fetch requests - it has no effect if specified on any other kind 
// of request.
//
// @see dataSource.progressiveLoading
// @see operationBinding.progressiveLoading
// @group progressiveLoading
// @visibility external
//<


// ServerObject
// ---------------------------------------------------------------------------------------
//> @attr DataSource.serverObject (ServerObject : null : IR)
//
// For Direct Method Invocation (DMI) binding, declares the ServerObject to use as the default
// target for all +link{DataSource.operationBindings}.  Specifying this
// attribute in an XML DataSource stored on the server enables DMI for this DataSource.
//
// @group serverDataIntegration
// @requiresModules SCServer
// @serverDS allowed
// @visibility external
//<

//> @class ServerObject
//
// The ServerObject tells the ISC server how to find or create a server-side object involved in
// +link{dmiOverview,DMI} (Direct Method Invocation).  
// <p>
// A ServerObject declaration appears in the XML definition of a +link{DataSource} (for
// responding to +link{DSRequest}s) or in an Application configuration file (.app.xml) for
// responding to +link{RPCRequest}s.
// <P>
// NOTE: Please take note of the points made in 
// +link{group:serverDataSourceImplementation,this discussion} of caching and thread-safety
// issues in server-side DataSources.
//
// @see DMI
//
// @treeLocation Client Reference/RPC
// @requiresModules SCServer
// @serverDS only
// @visibility external
//<

//> @attr serverObject.lookupStyle ("new"|"factory"|"attribute"|"spring" : "new" : IR)
//
// Specifies the mechanism for locating the class instance on which to invoke the method.
// Valid values are as follows:
// <ul>
// <li>"spring": For use with the +externalLink{http://springframework.com,Spring framework}.
// +link{serverObject.bean} contains the name of the bean to invoke.  Which application context
// is used can be configured via web.xml (see the example web.xml in the SDK).  See also
// +link{group:serverInit} for special concerns with framework initialization when using
// Spring.
// <li>"new": A new instance of the class specified by +link{serverObject.className} will be
// created and the DMI method will be invoked on that instance (unless the specified method is
// static, in which case no instance is created, but the class specified by
// +link{serverObject.className} is still used).
// <li>"factory": A custom factory provides the class instance on which the DMI method is to be
// invoked.  In this case, +link{serverObject.className} specifies the className of the factory
// that will provide the instance on which the DMI method is to be invoked.  The class
// specified by +link{serverObject.className} must provide exactly one method named
// <code>create</code> that must return the class instance on which you wish the DMI method to
// be invoked.  Like the DMI methods, the <code>create</code> method can request a standard set of
// values as arguments.  See +link{dmiOverview,DMI} for a list of available values.
// <li> "attribute": The instance on which the DMI method is to be invoked is looked up in the
// scope defined by +link{serverObject.attributeScope} via the attribute name specified in
// +link{serverObject.attributeName}.
// </ul>
//
// @see serverObject.className
// @see serverObject.attributeName
// @see serverObject.attributeScope
//
// @serverDS only
// @visibility external
//<

//> @attr serverObject.className (String : null : IR)
//
// Specifies the fully-qualified class name that provides the server-side endpoint of the DMI
// (+link{serverObject.lookupStyle}:"new") or the class name of the factory that produces the
// DMI instance (+link{serverObject.lookupStyle}:"factory").
// <p>
// This is one of the values that you need to pass to +link{DMI.call} to invoke the DMI from
// the client.
// <p>
// The value of this attribute is used only for <code>"new"</code> and <code>"factory"</code>
// values of +link{serverObject.lookupStyle}.
//
// @see serverObject.lookupStyle
// @see serverObject.ID
// @see DMI.call
//
// @serverDS only
// @visibility external
//<

//> @attr serverObject.methodName (String : null : IR)
//
// Specifies the name of the method to call for operations using this ServerObject.  This is 
// a DataSource-level default; you can override it for individual operations either by 
// specifying the +link{operationBinding.serverMethod} attribute, or by declaring an 
// operation-level serverObject that specifies a different methodName (if you specify both 
// an operationBinding.serverMethod and an operation-level serverObject.methodName, the 
// latter takes precedence)
//
// @serverDS only
// @visibility external
//<

//> @attr serverObject.ID (String : null : IR)
//
// You can optionally specify an ID on the ServerObject config block - in which case you can
// use that value as the "className" argument when calling +link{DMI.call}.  This allows you to
// hide the name of the server-side class used as the factory or implementer of the DMI from
// the browser as a security precaution.
//
// @see serverObject.className
// @see DMI.call
//
// @serverDS only
// @visibility external
//<

//> @attr serverObject.bean (String : null : IR)
//
// For use when +link{serverObject.lookupStyle} is <code>"spring"</code>, id of the bean to ask
// Spring to create.
//
// @serverDS only
// @visibility external
//<

//> @attr serverObject.attributeName (String : null : IR)
//
// Specifies the name of the attribute by which to look up the DMI instance.  This attribute is
// consulted only when the value of +link{serverObject.lookupStyle} is <code>"attribute"</code>.
//
// @see serverObject.attributeScope
// @see serverObject.lookupStyle
//
// @serverDS only
// @visibility external
//<

//> @attr serverObject.attributeScope ("request"|"session"|"application" : null : IR)
//
// Specifies the scope in which the DMI instance is to be looked up.  Valid values are: 
// <code>"request"</code>, <code>"session"</code>, and <code>"application"</code>.  If
// <code>attributeScope</code> is left out of the <code>ServerObject</code> definition, then
// all scopes are searched in the order in which they are listed above.
// <p>
// This attribute is consulted only when the value of +link{serverObject.lookupStyle} is
// <code>"attribute"</code>.
//
// @see serverObject.attributeName
// @see serverObject.lookupStyle
//
// @serverDS only
// @visibility external
//<


//> @attr serverObject.visibleMethods (List : null : IR)
//
// When the +link{ServerObject} appears in a .app.xml file (for RPC DMI), this property
// specifies the list of methods on the ServerObject that are callable from the client.  See
// the builtin.app.xml file in the /shared/app directory of the SDK for an example of a
// visibleMethods declaration block.
//
// @see DMI
//
// @serverDS only
// @visibility external
//<


//> @attr serverObject.dropExtraFields     (Boolean : null : IR)
// By default, for DMI DSResponses, DSResponse.data is filtered on the server to just the set
// of fields defined on the DataSource.  This behavior can be overridden in several ways - see
// the overview in +link{dmiOverview,DMI} for details.  The value of this attribute overrides
// +link{DataSource.dropExtraFields}.
// 
// @serverDS only
// @visibility external
//<


//> @attr serverObject.targetXPath     (String : null : IR)
// If set, the SmartClient server will use JXPath to call your server code.  The
// <code>JXPathContext</code> (start point) will be the object arrived at by applying the 
// +link{serverObject.lookupStyle,lookupStyle} and related ServerObject properties. The 
// intention of this property is to allow easier access to your existing Java objects 
// and reduce the need to write SmartClient-specific server code.
// 
// @serverDS only
// @visibility external
//<

//> @attr serverObject.crudOnly (boolean : null : IR)
//
// For a ServerObject defined at the +link{DataSource.serverObject,DataSource level}, by 
// default we only allow it to intercept standard CRUD operations (ie, ordinary fetches, adds,
// updates and removes).  To allow the ServerObject to intercept other types of operation - 
// custom operations, validations, etc - set this property to false.  Note that ServerObjects 
// declared at the +link{OperationBinding.serverObject,OperationBinding level} always intercept
// that operation, whatever its type, and this property has no effect.
// <p>
// <b>NOTE:</b> If you are intercepting operations on the server because you wish to inspect
// them before deciding whether to process them with bespoke code or allow them to proceed 
// with normal processing, the way to invoke normal processing without causing any interference
// is:<pre>
//    return dsRequest.execute();
// </pre>
//
// @requiresModules SCServer
// @serverDS only
// @visibility external
//<

// OperationBindings
// ---------------------------------------------------------------------------------------

//> @attr DataSource.operationBindings (Array of OperationBinding : null : IR)
// Optional array of OperationBindings, which provide instructions to the DataSource about how each
// DSOperation is to be performed.
// <P>
// When using the SmartClient Server, OperationBindings are specified in your DataSource
// descriptor (.ds.xml file) and control server-side behavior such as what Java object to route
// DSRequest to (+link{operationBinding.serverObject}) or customizations to SQL / HQL queries
// (+link{operationBinding.customSQL} and +link{operationBinding.customHQL}).  See the 
// +explorerExample{javaDataIntegration,Java Integration samples}.
// <P>
// For DataSources bound to WSDL-described web services using
// +link{DataSource.serviceNamespace}, OperationBindings are used to bind each DataSource
// +link{operationBinding.operationType,operationType} to an
// +link{operationBinding.wsOperation,operation} of a WSDL-described 
// +link{WebService,web service}, so that a DataSource can both fetch and save data to a web
// service.
// <P>
// For example, this code accomplishes part of the binding to the 
// +externalLink{http://www.google.com/search?q=sforce+partner+wsdl,SalesForce partner web services}
// <pre>
// isc.DataSource.create({
//    serviceNamespace : "urn:partner.soap.sforce.com",
//    operationBindings : [
//        { operationType:"fetch", wsOperation:"query", recordName: "sObject" },
//        { operationType:"update", wsOperation:"update", recordName: "SaveResult" },
//        { operationType:"add", wsOperation:"create", recordName: "SaveResult" },
//        { operationType:"remove", wsOperation:"delete", recordName: "DeleteResult" }
//    ],
//    ...
// }); 
// </pre>
// NOTE: additional code is required to handle authentication and other details, see the
// complete code in smartclientSDK/examples/databinding/SalesForce.
// <P>
// For DataSources that contact non-WSDL-described XML or JSON services, OperationBindings can
// be used to separately configure the URL, HTTP method, input and output processing for each
// operationType.  This makes it possible to fetch JSON data from one URL for the "fetch"
// operationType and save to a web service for the "update" operationType, while appearing as a
// single integrated DataSource to a +link{DataBoundComponent} such as an
// +link{ListGrid.canEdit,editable ListGrid}.
// <P>
// If no operationBinding is defined for a given DataSource operation, all of the properties
// which are valid on the operationBinding are checked for on the DataSource itself.  
// <P>
// This also means that for a read-only DataSource, that is, a DataSource only capable of fetch
// operations, operationBindings need not be specified, and instead all operationBinding
// properties can be set on the DataSource itself.  An example of using OperationBinding
// properties directly on the DataSource in order to read an RSS feed can be found here:
// <P>
// +standaloneExample{/examples/databinding/rss_databinding.html}
// 
// @see class:OperationBinding
// @serverDS allowed
// @visibility xmlBinding
//<

//> @class OperationBinding
// An operationBinding tells a DataSource how to execute one of the basic DS operations: fetch,
// add, update, remove.  See +link{dataSource.operationBindings}.
//
// @treeLocation Client Reference/Data Binding/DataSource
// @serverDS allowed
// @visibility xmlBinding
//<

//> @attr operationBinding.operationType (DSOperationType : null : IR)
// Which operationType this operationBinding is for.  This property is only settable on an
// operationBinding, not a DataSource as a whole.
//
// @group clientDataIntegration
// @serverDS allowed
// @visibility xmlBinding
//<

//> @attr operationBinding.operationId (String : null : IR)
// Optional operationId if this DataSource supports two or more variants of one of the basic
// DataSource operations, for instance, a "fetch" that uses full text search and a "fetch" that
// accepts per-field search criteria.  See +link{dsRequest.operationId} for usage.
//
// @group clientDataIntegration
// @serverDS allowed
// @visibility xmlBinding
//<

//> @attr operationBinding.serverMethod (string : null : IR)
//
// The name of the method to invoke on the +link{ServerObject} for this operationBinding.
// <p>
// <b>NOTE:</b> If you have a +link{DataSource.serverObject,DataSource-level ServerObject} and
// wish to override this operation so that it simply calls a different method on the same 
// server object, it is sufficient to specify just this property on the operationBinding:
// there is no need to redefine the serverObject at the operationBinding level.
//
// @requiresModules SCServer
// @serverDS only
// @visibility external
//<

//> @attr operationBinding.methodArguments (string : null : IR)
//
// You can explicitly declare the arguments to be passed to
// +link{operationBinding.serverMethod} using this attribute.  This isn't required - in the
// absence of <code>methodArguments</code>, the DMI implementation will still automatically
// pass a stock set of arguments to your method (see the overview in +link{ServerObject}), but
// specifying arguments gives you the ability to call pre-existing methods without adding
// SmartClient-specific code.
// <p>
// The format for specifying <code>methodArguments</code> is as a comma separated list of VTL
// (Velocity Template Language) expressions.  See the
// +externalLink{http://jakarta.apache.org/velocity/docs/vtl-reference-guide.html, VTL Reference}
// and +externalLink{http://jakarta.apache.org/velocity/docs/user-guide.html, Velocity User Guide} 
// for an overview of how to use VTL.
// <p>
// The Velocity context is pre-populated with the following variables - you can pass these
// verbatim as arguments, or call methods on these objects and pass the resulting values:
// <ul>
// <li>dsRequest: instance of the current DSRequest
// <li>request: the current HttpServletRequest
// <li>response: the current HttpServletResponse
// <li>rpcManager: the instance of RPCManager for this request
// <li>dataSource: a DataSource instance for this request
// </ul>
// So, for example, if you had a method signature like the following:
// <p><code>
// public DSResponse fetch(SupplyItem criteria, long startRow, long endRow)
// </code><p>
// You can invoke it by specifying <code>methodArguments</code> as follows:
// <p><code>
// methodArguments="$dsRequest.criteria, $dsRequest.startRow, $dsRequest.endRow"
// </code></p>
// Without <code>methodArguments</code>, there would be no way for you to specify
// <code>startRow/endRow</code> as arguments.  You could, of course, simply declare the method
// to take a <code>DSRequest</code> object and call <code>getStartRow()/getEndRow()</code> in
// the body of the method.
//
// @see ServerObject
//
// @serverDS only
// @visibility external
//<

//> @attr operationBinding.serverObject (ServerObject : null : IR)
//
// Optional ServerObject declaration that specifies the ServerObject configuration for this
// operationBinding.  In the absence of a serverObject specification here, the one specified on
// the DataSource itself is used by default, if present (+link{dataSource.serverObject}).  If
// neither is present, then Direct Method Invocation will not be enabled for this operationBinding.
//
// @see dataSource.serverObject
// @requiresModules SCServer
// @serverDS only
// @visibility external
//<

//> @attr operationBinding.script (String : null : IR)
// Scriptlet to be executed prior to the DataSource operation which is configured by this
// operationBinding.
// <P>
// Scriptlets are used similarly to DMIs configured via +link{operationBinding.serverObject} -
// they can add business logic by modifying the DSRequest before it's executed, modifying the
// default DSResponse, or taking other, unrelated actions.  
// <P>
// For example:
// <pre>
//    &lt;operationBindings&gt;
//       &lt;operationBinding operationType="add"&gt;
//           &lt;script language="groovy"&gt;
//              ... Groovy code ...
//           &lt;/script&gt;
//       &lt;/operationBinding&gt;
//    &lt;/operationBindings&gt;
// </pre>
// <P>
// Scriptlets can be written in any language supported by the "JSR 223" standard, including Java
// itself.  See the +link{group:dmiScript,DMI Script Overview} for rules on how to return data,
// add additional imports, and other settings.
// <P>
// The following variables are available for DMI scriptlets:
// <ul>
// <li><i>dataSource</i>: the current DataSource
// <li><i>dsRequest</i>: the current DSRequest
// <li><i>criteria</i>: shortcut to DSRequest.getCriteria() (a Map)
// <li><i>values</i>: shortcut to DSRequest.getValues() (a Map)
// <li><i>oldValues</i>: shortcut to DSRequest.getOldValues() (a Map)
// <li><i>sqlConnection</i>: <b>SQLDataSource only</b>: the current SQLConnection object.  If
// using +link{dataSource.autoJoinTransactions,automatic transactions} are enabled, this
// SQLConnection is in the context of the current transaction.
// <li><i>beanFactory</i>: the spring BeanFactory (when applicable)
// </ul>
// <P>
// Scriptlets also have access to a set of contextual variables related to the Servlets
// API, as follows:
// <ul>
// <li><i>servletRequest</i>: the current ServletRequest
// <li><i>session</i>: the current HttpSession
// <li><i>rpcManager</i>: the current RPCManager
// <li><i>servletResponse</i>: the current ServletResponse <b>(advanced use only)</b>
// <li><i>servletContext</i>: the current ServletContext<b>(advanced use only)</b>
// </ul>
// As with DMI in general, be aware that if you write scriptlets that depend upon these variables,
// you preclude your DataSource from being used in the widest possible variety of
// circumstances.  For example, adding a scriptlet that relies on the <code>HttpSession</code>
// prevents your DataSource from being used in a command-line process.
//
// @example scriptingUserSpecificData
// @serverDS only
// @visibility external
//<

// Input Handling
// ---------------------------------------------------------------------------------------

//> @attr operationBinding.wsOperation (String : null : IR)
// Name of the web service operation that will be invoked in order to execute this DataSource
// operation.
// <P>
// Valid only for a DataSource that has a WebService (+link{attr:dataSource.serviceNamespace} is
// set).  Otherwise, use +link{attr:operationBinding.dataURL}.
// <P>
// Setting <code>wsOperation</code> means that +link{dsRequest.data} will be serialized as the
// request message for the specified web service operation, with namespacing and soap encoding
// handled automatically.  See +link{DataSource.transformRequest()} for how to customize what
// data is sent to the server.
//
// @group clientDataIntegration
// @visibility xmlBinding
// @example wsdlBinding
//<

//> @attr operationBinding.dataURL              (URL : null : [IR])
// URL to contact to fulfill DSRequests for this operationBinding.
// <P>
// <code>dataURL</code> is typically set as DataSource.dataURL rather than on each individual
// operationBinding.
// <P>
// <code>dataURL</code> can be omitted for a DataSource using a Web Service
// (+link{attr:dataSource.serviceNamespace} is set).
//
// @group clientDataIntegration
// @serverDS allowed
// @visibility xmlBinding
//<

//> @attr operationBinding.dataProtocol            (DSProtocol : "getParams" : [IR])
// Controls the format in which inputs are sent to the dataURL.
// <p>
// When a DataSource operation such as fetchData() is invoked on this DataSource or a component
// bound to this DataSource, the data passed to the operation, if any, will be sent to the
// <code>dataURL</code>.  The <code>dataProtocol</code> property controls the format in
// which the data is sent: SOAP message, HTTP GET or POST of parameters, etc.
// <P>
// The <code>dataProtocol</code> property need not be set for a DataSource with a WebService
// ( +link{attr:dataSource.serviceNamespace} is set), in this case, SOAP messaging is used by
// default.
// <P>
// Developers may completely bypass the SmartClient comm system by setting dataProtocol to
// <code>"clientCustom"</code>. In this case SmartClient will not attempt to send any data
// to the server after calling +link{DataSource.transformRequest()}. Instead the developer is
// expected to implement <code>transformRequest()</code> such that it performs the necessary
// data action outside of SmartClient, and then calls +link{DataSource.processResponse()}, 
// passing in the +link{DSRequest.requestId} and an appropriate set of DSResponse properties to
// indicate the result of the action.
// <P>
// NOTE: when +link{dataFormat} is "iscServer", <code>dataProtocol</code> is not consulted.
// Instead, SmartClient uses a proprietary wire format to communicate with the SmartClient
// server, and the server-side DSRequest and DSResponse objects should be used to access
// request data and form responses.
//
// @see type:DSProtocol
// @group clientDataIntegration
// @serverDS allowed
// @visibility xmlBinding
//<
//dataProtocol:"getParams", // NOTE: dynamic default determined in getServiceInputs()

//> @attr operationBinding.dataFormat (DSDataFormat : "iscServer" : [IR])
// Format for response data for this operation.
// <P>
// Typically set once for the DataSource as a whole via +link{DataSource.dataFormat}.
//
// @group clientDataIntegration
// @visibility xmlBinding
//<

//> @attr operationBinding.dataTransport      (RPCTransport : RPCManager.defaultTransport : [IR])
// Transport to use for this operation.  Defaults to +link{DataSource.dataTransport}, which in
// turn defaults to +link{RPCManager.defaultTransport}.  This would typically only be set to
// enable "scriptInclude" transport for contacting +link{DataSource.dataFormat,JSON} web
// services hosted on servers other than the origin server.
// <p>
// When using the "scriptInclude" transport, be sure to set +link{dataSource.callbackParam} or
// +link{operationBinding.callbackParam} to match the name of the query parameter name expected
// by your JSON service provider.
//
// @see RPCTransport
// @see dataSource.callbackParam
// @group clientDataIntegration
// @visibility xmlBinding
//<

//> @attr operationBinding.useHttpProxy (boolean : null : IR)
// Whether to use the +link{RPCManager.sendProxied,HttpProxy} servlet to send requests
// described by this operationBinding.  If unset, automatically detects whether using the
// HttpProxy is necessary based on the same-origin policy.
// <P>
// Valid only with +link{dataProtocol} settings other than ISCServer.
//
// @group clientDataIntegration
// @visibility external
//<

//> @attr operationBinding.callbackParam      (String : "callback" : [IR])
// Applies only to dataFormat: "json".  Specifies the name of the query parameter that
// tells your JSON service what function to call as part of the response for this operation.
// <P>
// Typically set once for the DataSource as a whole via +link{DataSource.callbackParam}.
//
// @see dataSource.callbackParam
// @group clientDataIntegration
// @serverDS allowed
// @visibility xmlBinding
//<


//> @attr operationBinding.requestProperties (DSRequest Properties : null : [IR])
// Additional properties to pass through to the +link{DSRequest} created for this operation.
// Note that these will be cumulative with and will override on a per-property basis any
// properties set via +link{dataSource.requestProperties}.
// <p>
// These properties are applied before +link{dataSource.transformRequest} is called.
//
// @see DSRequest
// @see dataSource.requestProperties
// @group clientDataIntegration
// @group serverDataIntegration
// @serverDS allowed
// @visibility external
//<

//> @attr operationBinding.preventHTTPCaching (boolean : null : IR)
// Configures +link{dataSource.preventHTTPCaching} on a per-operationType basis.
// @serverDS allowed
// @visibility external
//<

//> @type DSProtocol
// +link{OperationBinding.dataProtocol} affects how the data in the DSRequest (+link{dsRequest.data})
// is sent to the +link{dataSource.dataURL}.  Listed below are the valid values for
// +link{OperationBinding.dataProtocol} and their behavior.
// <P>
// Note that, when using the SmartClient server, data is automatically translated from
// JavaScript to Java according to the rules described +link{rpcRequest.data,here};
// dataProtocol does not apply and is ignored.
// <P>
// If you are integrating with a +link{RestDataSource,REST} server that requires the more
// obscure +link{rpcRequest.httpMethod}s of "PUT", "DELETE" or "HEAD", you can specify these
// httpMethod settings via +link{operationBinding.requestProperties}.  dataProtocol settings
// that mention "GET" or "POST" are compatible with these additional HTTP methods as well.
// Typical +link{dataSource.operationBindings,operationBindings} for a REST server that uses
// "PUT" and "DELETE" are as follows:
// <pre>
//    operationBindings:[
//       {operationType:"fetch", dataProtocol:"getParams"},
//       {operationType:"add", dataProtocol:"postParams"},
//       {operationType:"remove", dataProtocol:"getParams", requestProperties:{httpMethod:"DELETE"}},
//       {operationType:"update", dataProtocol:"postParams", requestProperties:{httpMethod:"PUT"}}
//    ],
// </pre>
// 
// @value "getParams"
//   Data is added to the dataURL, with each property in the data becoming an HTTP parameter,
//   eg http://service.com/search?keyword=foo
// @value "postParams"
//   Data is POST'd to the dataURL, with each property becoming an HTTP parameter, 
//   exactly as an HTML form would submit them if it had one input field per property in the
//   data.
// @value "postXML"
//   Data is serialized as XML via +link{dataSource.xmlSerialize()} and POST'd as the
//   HTTP request body with contentType "text/xml".
// @value "soap"
//   Data is serialized as XML via +link{dataSource.xmlSerialize()}, wrapped in a SOAP
//   envelope, and POST'd as the HTTP request body with contentType "text/xml".  Generally
//   only used in connection with a +link{group:wsdlBinding,WSDL web service}.
// @value "postMessage"
//   dsRequest.data is assumed to be a String set up by +link{dataSource.transformRequest()}
//   and is POST'd as the HTTP request body.
// @value "clientCustom"
//   This setting entirely bypasses the SmartClient comm system. Instead of the DataSource sending
//   an HTTP request to the server, the developer is expected to implement
//   +link{DataSource.transformRequest()} to perform their own custom data manipulation logic, and
//   then call +link{DataSource.processResponse()} to handle the results of this action.
//   
//
// @see operationBinding.dataProtocol
// @group clientDataIntegration
// @serverDS allowed
// @visibility xmlBinding
//<

//> @attr operationBinding.defaultParams    (Object : null : [IRA]) 
// HTTP parameters that should be submitted with every DSRequest.
// <P>
// Useful for authenticated services that require a sessionId with every request.
// <P>
// Can be set for all operations of a given DataSource as DataSource.defaultParams.
//
// @group clientDataIntegration
// @serverDS allowed
// @visibility xmlBinding
//<

//> @attr operationBinding.useFlatFields          (boolean : false : [IR])
// Setting <code>useFlatFields</code> on an operationBinding is equivalent to setting
// +link{dsRequest.useFlatFields} on all DataSource requests with the same +link{operationType}
// as this <code>operationBinding</code>.
// <P>
// Typical usage is to combine operationBinding.useFlatFields with 
// +link{dataBoundComponent.useFlatFields,searchForm.useFlatFields}, with the +link{SearchForm}
// bound to the +link{webService.getInputDS(),input message} of the web service operation set
// as +link{operationBinding.wsOperation}.  This allows gratuitous nesting to be consistently
// bypassed in both the user presentation and in the actual XML messaging.
// <P>
// Note that <code>useFlatFields</code> is not generally recommended for use with input messages
// where multiple simple type fields exist with the same name, however if used in this way,
// the first field to use a given name wins.  "first" means the first field encountered in a
// depth first search.  "wins" means only the first field will be available in data binding,
// and only the first field will be populated in the generated XML message.
//
// @group clientDataIntegration
// @serverDS allowed
// @visibility xmlBinding
//<


// XML Output handling
// ---------------------------------------------------------------------------------------

//> @attr operationBinding.recordXPath          (XPathExpression : null : [IR])
// For an XML or JSON DataSource, XPath expression used to retrieve the objects that will
// become DataSource records.
// <p>
// For example, an "ItemSearch" web service might return a "Results" structure containing
// metadata along with the set of Items that one might want to display in a grid.  An XPath
// expression like "/Results/Items" could be used to retrieve just the Items, which would then
// become DataSource records.
// <p>
// For a JSON web service, the <code>recordXPath</code> is applied to the returned JSON data
// via +link{XMLTools.selectObjects()}.  Only limited XPath syntax is allowed; see 
// +link{XMLTools.selectObjects(),selectObjects()} for details.
// <P>
// For processing XML results, see +link{attr:operationBinding.xmlNamespaces} for information
// on the namespaces that are available in this XPath expression.  If you are contacting a WSDL
// web service, note that +link{operationBinding.recordName} is an alternative way to specify
// which records should be selected by their tagName or type, and this is usually simpler.
// <P>
// To learn about XPath, try the following search:
// <a href="http://www.google.com/search?q=xpath+tutorial" target="_blank"
// >http://www.google.com/search?q=xpath+tutorial</a>
//
// @group clientDataIntegration
// @serverDS allowed
// @visibility xmlBinding
//< 

//> @attr operationBinding.recordName           (String : null : [IR])
// For an XML DataSource, tagName of the elements to be used as records.
// <p>
// This is a simple alternative to +link{attr:recordXPath} when the elements to be used as
// records all share a tagName.
// <p>
// When a DataSource has a WebService, <code>recordName</code> can also be set to the name
// of any <code>complexType</code> declared within the WebService's WSDL file.
//
// @group clientDataIntegration
// @visibility xmlBinding
// @serverDS allowed
// @example wsdlBinding
//< 

//> @attr operationBinding.spoofResponses (boolean : false : IR)
// For a DataSource contacting a +link{dataSource.serviceNamespace,WSDL web service}, setting
// this flag means the DataSource doesn't actually attempt to contact the server but generates
// a sample response instead, based on the XML Schema of the response message embedded in the
// WSDL.
// <P>
// The spoofed response will include all complexType elements and will fill in appropriate
// values by type for all simpleType elements, although the spoofed data will not conform to
// all xs:restriction declarations (eg xs:pattern).
// <P>
// Note that if your WSDL does not fully describe the response format (some WSDL services just
// have a placeholder &lt;xs:any&gt; element), SmartClient can only produce a partial
// response.  To use a hand-generated sample response, just save an XML file to disk and use
// the +link{operationBinding.dataURL} setting to point to it.
//
// @group clientDataIntegration
// @serverDS allowed
// @visibility xmlBinding
//<

//> @attr operationBinding.xmlNamespaces              (Object : null : [IR])
// Optional object declaring namespace prefixes for use in +link{attr:recordXPath} and
// +link{attr:DataSourceField.valueXPath} XPath expressions.
// <P>
// <code>xmlNamespaces</code> should be specified as a mapping from namespace prefix to
// namespace URI, for example:
// <pre>
//    xmlNamespaces : {
//        az : "http://webservices.amazon.com/AWSECommerceService/2005-03-23"
//    }
// </pre>
// By default, all namespaces declared on the document element (outermost element of the
// response) are made available with the prefix used in the document itself.
// <P>
// Then, for non-WSDL-described XML results, if there is a default namespace on the document
// element, it is made available with the special prefix "default".
// <P>
// For results of WSDL-described operations, the prefix "service" means the service namespace,
// that is, the "targetNamespace" on the &lt;definitions&gt; element from the WSDL file.  The prefix
// "schema" means the namespace of the outermost element in the output message for the current
// operation.  "default" will be the schema namespace if there is one, otherwise the service
// namespace.
// <P>
// For basic information on XML Namespaces and their use in XPath, try the following search:
// <a href="http://www.google.com/search?q=XPath+xml+namespaces" target="_blank"
// >http://www.google.com/search?q=XPath+xml+namespaces</a>
//
// @group clientDataIntegration
// @visibility xmlBinding
// @serverDS allowed
// @example xmlYahooWebServices
//<

//> @attr operationBinding.responseDataSchema           (DataSource : null : [IRA])
// Optional schema describing how to extract DataSource records from the XML elements selected.
// <P>
// Once a set of XML elements have been selected via <code>recordXPath</code> or
// <code>recordName</code>, those elements are normally transformed to JavaScript objects
// using the <code>fields</code> of the DataSource that owns the operationBinding.  A
// <code>responseDataSchema</code> can be specified instead if the XML differs in some way between
// different DataSource operations, such that different values for
// +link{attr:DataSourceField.valueXPath,field.valueXPath} may be necessary to extract the same
// DataSource record from slightly different XML structures.
//
// @group clientDataIntegration
// @serverDS allowed
// @visibility xmlBinding
//<

// Export
// ---------------------------------------------------------------------------------------

//> @attr operationBinding.exportResults (boolean : false : IR)
// When set, causes the results of the DataSource Operation to be exported to a file, whose 
// name and format are indicated by +link{operationBinding.exportFilename} and 
// +link{operationBinding.exportAs} respectively.  When no exportFilename is provided, the 
// default is <i>Results</i> and the default value of exportAs is <i>csv</i>.  Once the Operation 
// completes, +link{dsRequest.exportDisplay} specifies whether the 
// exported data will be downloaded to the file-system or displayed in a new window.   The 
// default value of exportDisplay is "download" which displays the Save As dialog.  See 
// +link{ExportDisplay} for more information.
// <P>
// The export field-list can also be configured, see +link{dsRequest.exportFields}.
// <P>
// You can also configure the style of line-breaks to use when generating the output.  See
// +link{LineBreakStyle} for more information.
// <P>
// As well as setting this and other properties on the +link{OperationBinding}, Exports can be 
// initiated in two other ways.  You can set properties on the dsRequest by passing 
// <i>requestProperties</i> into +link{dataSource.exportData()}.  Note that this method does
// not support exporting to JSON format (see 
// <a href="http://forums.smartclient.com/showthread.php?t=235">this post</a> for more detail).
// Additionally, custom server code may set export-related properties on the 
// +link{DSResponse}.
// <P>
// <b>Format Examples</b>
// <P>
// XML format
// <pre>
//     &lt;List&gt;
//         &lt;Object&gt;
//             &lt;id&gt;10101&lt;/id&gt;
//             &lt;displayName&gt;Record 10101&lt;/displayName&gt;
//         &lt;/Object&gt;
//    &lt;/List&gt;
// </pre>
// JSON Format
// <pre>
//     [
//         { id: 10101, displayName: "Record 10101" }
//     ]
// </pre>
// CSV Format
// <pre>
//     id,displayName
//     10101,"Record 10101"
// </pre>
//
// @serverDS allowed
// @visibility external
//<

//> @attr operationBinding.exportAs (ExportFormat : "csv" : IR)
// The format in which the data should be exported.  Default is "csv".
// See +link{ExportFormat} for more information.
//
// @serverDS allowed
// @visibility external
//<

//> @attr operationBinding.exportFilename (String : null : IR)
// The name of the file to save the exported data into.
//
// @serverDS allowed
// @visibility external
//<

//> @attr operationBinding.lineBreakStyle (String : null : IR)
// The style of line-breaks to use in the exported output.  See +link{LineBreakStyle} for
// more information.
//
// @serverDS allowed
// @visibility external
//<

//> @attr operationBinding.exportFields (Array : null : IR)
// The list of field-names to export.  If provided, the field-list in the exported output is 
// limited and sorted as per the list.
// <P>
// If exportFields is not provided, the exported output includes all visible fields 
// from the DataSource (field.hidden=false), sorted in the order they're defined.
//
// @serverDS allowed
// @visibility external
//<

//> @attr operationBinding.autoJoinTransactions (boolean : null : IR)
// If true, causes requests against this operation to automatically start or join a transaction.
// if false, causes requests against this operation to be committed individually.  If null, 
// falls back to +link{DataSource.autoJoinTransactions}.
// <P>
// See +link{DataSource.autoJoinTransactions} for further details of SmartClient's automatic
// transaction control.
//
// @serverDS only
// @visibility transactions
//<

// Velocity template variables
// ---------------------------------------------------------------------------------------
//> @groupDef velocitySupport
// The SmartClient Server provides a number of standard context variables for use in the 
// Velocity templates you write to implement +link{group:customQuerying,custom queries}, 
// +link{dsRequestModifier.value,transaction chaining},  
// +link{operationBinding.requires,dynamic security checking} and 
// +link{object:Mail,templated mail messages}.  These are:
// <ul>
// <li><b>$currentDate</b>. The current date/time with millisecond precision</li>
// <li><b>$transactionDate</b>. The date/time that this transaction started, with millisecond 
// precision.  If you are not using +link{RPCManager.startQueue,queuing}, this value will be
// identical to <b>$currentDate</b></li>
// <li><b>$servletRequest</b>. The associated <code>HttpServletRequest</code></li> 
// <li><b>$dsRequest</b>. The associated +link{class:DSRequest} (though of course this is a
// server-side <code>DSRequest</code> object, so please also see the server-side Javadocs)</li> 
// <li><b>$primaryDSRequest</b>. Only present on cache-sync operations, this is the original
// update <code>DSRequest</code> that caused the cache-sync request to be created</li> 
// <li><b>$session</b>. The associated <code>HttpSession</code></li>
// <li><b>$httpParameters</b>. This variable gives you access to the parameters Map of the 
// associated <code>HttpServletRequest</code>; it is an alternate form of 
// <code>$servletRequest.getParameter</code></li>
// <li><b>$requestAttributes</b>. This variable gives you access to the attributes Map of the 
// associated <code>HttpServletRequest</code>; it is an alternate form of 
// <code>$servletRequest.getAttribute</code></li>
// <li><b>$sessionAttributes</b>. This variable gives you access to the attributes Map of the 
// associated <code>HttpSession</code>; it is an alternate form of 
// <code>$session.getAttribute</code></li>
// <li><b>$dataSources</b>. This variable gives you access to SmartClient +link{class:DataSource}s.  
// You access a dataSource by suffixing its name to the <code>$dataSources</code> designation.
// For example, <code>$dataSources.supplyItem</code> refers to the DataSource object called 
// "supplyItem".  You can use this approach to execute any valid DataSource method. One especially
// useful method in this context is <code>hasRecord(fieldName, value)</code> - see the 
// server-side Javadocs for more details.
// <li><b>$util</b> - A <code>DataTools</code> object, giving you access to all of that
//     class's useful helper functions</li>
// </ul>
// All of these variables (other than the two dates) represent objects that can contain other 
// objects (attributes or parameters or object properties).  The variables based on the Servlet
// API (session, sessionAttributes, httpParameters, servletRequest and requestAttributes) all 
// implement the <code>Map</code> interface, so you can use the Velocity "property" shorthand 
// notation to access them.  The following usage examples show five equivalent ways to return 
// the value of the session attribute named "foo":
// <pre>
//    $session.foo
//    $session.get("foo")
//    $session.getAttribute("foo")
//    $sessionAttributes.foo
//    $sessionAttributes.get("foo")
// </pre>
// In the case of <code>$servletRequest</code>, the shorthand approach accesses the attributes 
// - you need to use either <code>$httpParameters</code> or <code>$servletRequest.getParameter</code>
// to access parameters. These examples all return the value of the HTTP parameter named "bar":
// <pre>
//    $httpParameters.bar
//    $httpParameters.get("bar")
//    $servletRequest.getParameter("bar")
// </pre>
// When you use these Velocity variables in a +link{operationBinding.customSQL,customSQL} 
// clause or SQL snippet such as a +link{operationBinding.whereClause,whereClause}, all of 
// these template variables return values that have been correctly quoted and escaped according
//  to the syntax of the underlying database.  We do this because "raw" values are vulnerable to 
// <a href="http://en.wikipedia.org/wiki/SQL_injection">SQL injection attacks</a>.
// If you need access to the raw value of a variable in a SQL template, you can use the 
// <b>$rawValue</b> qualifier in front of any of the template variables, like this:<br><br><code>
// &nbsp;&nbsp;$rawValue.session.foo</code>
// <p>
// This also works for the <b>$criteria</b> and <b>$values</b> context variables (see 
// +link{group:customQuerying} for details of these variables).  So:<br><br><code>
// &nbsp;&nbsp;$rawValue.criteria.customerName</code>
// </p>
// Note that <code>$rawValue</code> is only available in SQL templates.  It is not needed in
// other contexts, such as Transaction Chaining, because the value is not escaped and 
// quoted in these contexts.
// <p>
// <b>Warning</b>:  Whenever you access a template variable for use in a SQL statement, bear 
// in mind that it is <b>dangerous</b> to use <code>$rawValue</code>.  There are some cases 
// where using the raw value is necessary, but even so, all such cases are likely to be vulnerable 
// to injection attacks.  Generally, the presence of <code>$rawValue</code> in a SQL template 
// should be viewed as a red flag.
// <p>
// Finally, some example usages of these values.  
// These +link{operationBinding.values,values} clauses set "price" to a value extracted from the 
// session, and "lastUpdated" to the date/time that this transaction started:<br><br><code>
// &nbsp;&nbsp;&lt;values fieldName="price" value="$session.somePrice" /&gt;<br>
// &nbsp;&nbsp;&lt;values fieldName="lastUpdated" value="$transactionDate" /&gt;
// </code><p>
// This whereClause selects some users based on various values passed in the criteria and 
// as HTTP parameters:<br><br><code>
// &nbsp;&nbsp;&lt;whereClause&gt;department = $httpParameters.userDept AND dob &gt;= $criteria.dateOfBirth&lt;/whereClause&gt;
// </code><p>
// This whereClause selects some users based on various values obtained from the 
// servletRequest's attributes, using a number of equivalent techniques for accessing the 
// attributes:<pre>
//   &lt;whereClause&gt;
//         department = $servletRequest.dept 
//     AND startDate &gt;= $requestAttributes.dateOfBirth 
//     AND salary &lt; $servletRequest.getAttribute("userSalary")
//   &lt;/whereClause&gt;
// </pre><p>
//
// @title Velocity context variables
// @visibility chaining
//<


// Custom Querying
// ---------------------------------------------------------------------------------------

//> @type DefaultQueryClause
// The Velocity variable names of the "pieces" of SQL that SmartClient generates to form a
// complete fetch or update query.  You can use these variables in you own custom queries 
// and query clause overrides to build on the SmartClient functionality.  See 
// +link{group:customQuerying} for a full discussion.
//
// @value "$defaultSelectClause"  The column names to select, for a fetch operation only
// @value "$defaultTableClause"   The table name(s) to select from or update
// @value "$defaultWhereClause"   The "where" condition, which will be derived from supplied 
//                                criteria or a primary key value, depending on the type of 
//                                operation
// @value "$defaultValuesClause"  The column names to update and the update values, for an
//                                update or add operation
// @value "$defaultOrderClause"   The column names to sort by, for a fetch operation only
//
// @group customQuerying
// @serverDS allowed
// @visibility customSQL
//<

//> @groupDef customQuerying
// <b>This feature, called "SQL Templating", is available with Power or better licenses
// only.</b> See <a href="http://smartclient.com/product">smartclient.com/product</a> for details.
// <p>
// The SmartClient server provides a number of ways to let you customize the SQL or Hibernate
// query it generates to fetch data from or update your database.  You can provide full 
// custom queries in either +link{OperationBinding.customSQL,SQL} or 
// +link{OperationBinding.customHQL,HQL}, or you can replace individual parts of the query 
// (+link{OperationBinding.whereClause,the WHERE clause}, for example) while letting
// SmartClient generate the rest. 
// <P>
// Full custom queries specified via &lt;customSQL&gt; provide complete flexibility, but 
// they cannot be used for automatic data paging; if you use a full custom query, all data
// returned by the query will be delivered to the client, which may be inefficient.
// To retain automatic data paging, implement your customizations by replacing just specific
// clauses of the query, via +link{operationBinding.whereClause,&lt;whereClause&gt;}, 
// +link{operationBinding.selectClause,&lt;selectClause&gt;}, and the other clause-by-clause
// replacement features.
// <p>
// Query customization is done per +link{OperationBinding}, so you can create multiple
// customized queries per DataSource or even for the same 
// +link{OperationBinding.operationType,operation type}.
//
// <h4>Using criteria and values</h4>
// Whether using full custom queries or individual clauses, your code has access to the 
// criteria values submitted with the operation; if this is an "add" or "update" operation,
// it also has access to the new field values sent from the client.
// <p>
// Fields are accessed in your SQL or HQL code using the Velocity template language.  You
// can refer to container variables <b>$criteria</b> and <b>$values</b> in your queries or 
// clause snippets, and SmartClient will insert the appropriate values.  A simple 
// +link{OperationBinding.whereClause,whereClause} example:
// <p>
// <pre><code>
//  &lt;operationBinding operationType="fetch"&gt;
//     &lt;whereClause&gt;
//         continent = $criteria.continent AND population &gt; $criteria.minPop
//     &lt;/whereClause&gt;
//  &lt;/operationBinding&gt;
// </code></pre>
// In addition to the $criteria and $values Velocity template variables described above, we
// also provide a number of template variables containing generally-useful values.  Please see
// +link{group:velocitySupport} for details.
//
// <h4>Using the default clauses</h4>
// You also have access to the +link{type:DefaultQueryClause,default subclauses} generated by 
// SmartClient.  You can use these in full custom queries to allow a certain part of the query
// code to be generated:<p>
// <pre>
// &lt;customSQL&gt;
//    SELECT foo, bar FROM $defaultTableClause 
//        WHERE baz > $criteria.baz
// &lt;/customSQL&gt;
// </pre>
// <p>
// You can also use them within individual clauses in order to customize a clause without
// losing default SQL generation:
// <pre>
// &lt;whereClause&gt;
//    ($defaultWhereClause) AND foo > 5
// &lt;/whereClause&gt;
// </pre>
//
// <h4>Mixing SQL Templating and custom Java Logic</h4>
// You can use both SQL Templating and custom Java logic added via +link{DMI} in the same
// operationBinding.  Your DMI method is called before SQL is generated, and the SQL template
// will be evalauted and the actual SQL operation performed only when you call
// dsRequest.execute().
// <P>
// This allows you to modify the criteria or values on the DSRequest, which will change the
// values retrieved by $criteria and $values when the SQL Template is evaluated.  You can also
// add entirely new information to the Velocity context used to evaluate the template, via
// the server-side API DSRequest.addToTemplateContext().
//
// <h4>Excluding fields from SQL Generation</h4>
// In some cases you pass a value to the server which is intended to be used by custom Java
// logic and should not cause SQL to be generated.  To prevent all SQL from being generated for
// a particular field, set +link{dataSourceField.customSQL,customSQL="true"} on that field.  
// <P>
// Any field for which SQL will ever be generated must be declared in a DataSource.  It's
// common to have a field which is only used in one or two operationBindings - in this case,
// set customSQL="true" on the field, and use +link{operationBinding.customFields} to cause
// specific operationBindings to generate SQL for the field, while all others ignore it.
// <P>
// In other cases you want to hand-write SQL for a particular field for a specific
// operationBinding.  You can set +link{operationBinding.excludeCriteriaFields} to 
// exclude fields from SQL generation for the whereClause of a specific operationBinding.
//
// <h4>Field-level SQL Customization</h4>
// An individual field can configured with custom expressions to be used in different SQL
// statements for all operationBindings - see +link{dataSourceField.customSQL} for an overview.
// 
// <h4>Using AdvancedCriteria</h4>
// The above examples involving $criteria assume that the submitted criteria are simple
// Criteria and not +link{AdvancedCriteria}, a more sophisticated criteria format in which
// different search operators can be specified per field and criteria can be nested.  
// <P>
// The special variable $advancedCriteria provides simplified access to the AdvancedCriteria
// structure: $advancedCriteria.<i>fieldName</i> will return the criteria value specified for a
// given fieldName, regardless of where it's present in the AdvancedCriteria.
// <P>
// This makes it straightforward to add an additional criteria value to AdvancedCriteria that
// you want to use only in the SQL template:
// <ul>
// <li> make a simple Criteria object representing the fieldName and value name you need to have
// available in the SQL template
// <li> use +link{DataSource.combineCriteria()} to add your additional criteria to an existing
// AdvancedCriteria, wherever this is convenient
// <li> list the fieldName in +link{operationBinding.customCriteriaFields} to prevent the
// default SQL for this field from being generated
// <li> use $advancedCriteria in your customized SQL to access the value
// </ul>
// Java API dsRequest.getCriteriaValue() is equivalent to accessing $advancedCriteria in a SQL
// Template.  Also note, if a given fieldName appears more than once in AdvancedCriteria,
// $advancedCriteria.<i>fieldName</i> will return the value for the first +link{Criterion} that
// uses the fieldName, as found by depth-first search.
// <p>
// NOTE: $advancedCriteria falls back to simple criteria values if the current criteria object 
// is not an <code>AdvancedCriteria</code>.  This means that you can safely use $advancedCriteria
// in circumstances where you cannot predict in advance whether your server code will be handed
// a simple criteria or an AdvancedCriteria.
//
// <h4>Stored procedures</h4>
// It is possible to include templated calls to SQL stored procedures in a
// +link{OperationBinding.customSQL,customSQL} clause, for the ultimate in flexibility.  For 
// example, the deletion of an order might require a number of actions: deletion of the order
// record itself, messages sent to other systems (data warehousing, maybe, or a central accounts
// system running on a mainframe), an event log written, and so on.  You could write a stored 
// procedure to do all this, and then invoke it with a customSQL clause:
// <pre>
//    &lt;operationBinding operationType="remove"&gt;
//        &lt;customSQL&gt;call deleteOrder($criteria.orderNo)&lt;/customSQL&gt;
//    &lt;/operationBinding&gt;
// </pre>
// When calling stored procedures this way, be sure that the &lt;customSQL&gt; operates like a
// normal SQL statement, so that it can be called via normal JDBC calls.  For
// operationType="fetch", the JDBC API PreparedStatement.executeQuery() is called and expects a
// ResultSet returned.  For "update", "add" and "remove" operationTypes,
// PreparedStatement.executeUpdate() is called and expects an integer return value (number of
// rows affected).  If your stored procedure uses in/out parameters, returns something other
// than a ResultSet or affected row count, or in some other way is incompatble with the
// standard JDBC methods described above, you may need to add code to adjust the return values
// to what JDBC expects.  You can do this by either putting code into the &lt;customSQL&gt;
// block directly, or by adding a second stored procedure that transforms the outputs of the
// first procedure.
//
// <h4>Velocity Template Language conditional logic</h4>
// When writing a customized SQL clause for an operation, it is commonly desirable to be
// able to include conditional logic - for example only modifying a where clause if a
// certain criteria value is present. Velocity template language conditional statements
// can be embedded directly into your template code to achieve this. For example the following
// <code>whereClause</code> would produce different output depending on whether the 
// request criteria included a value for the field <code><i>someField</i></code>:<p>
// <code>&lt;whereClause&gt;$defaultWhereClause #if ($criteria.someField) AND someDatabaseField = $criteria.someField #end&lt/whereClause&gt;</code>
// <p>
// If <code><i>criteria.someField</i></code> was not present in the request, the generated
// SQL statement would simply use the default where clause -- otherwise
// <code>AND someDatabaseField = <i>[some value]</i></code> would be appended to it (where
// <code><i>[some value]</i></code> was picked up from the value of <code>someField</code> on 
// the request criteria object).
//
// <h4>Custom queries are safe</h4>
// Custom queries are protected from <a href=http://en.wikipedia.org/wiki/SQL_injection>
// SQL injection attacks</a>, because anything coming from the client is quoted and escaped 
// in accordance with the syntax of the underlying database before use (though see the warning
// about using <code>$rawValue</code> in the article on +link{velocitySupport}).
// So, in a typical SQL injection attack an attacker might enter his User ID as <br>
// &nbsp;&nbsp;<code>123' OR '1' = '1</code><p>
// in the hope that this will generate a query
// with a where clause like this<br>
//  &nbsp;&nbsp;<code>WHERE userID = '123' OR '1' = '1'</code><p>
// which would of course return every row.  With SmartClient custom queries, this does not happen; 
// the client-provided string is escaped, and the resultant clause would look like this: <br>
// &nbsp;&nbsp;<code>WHERE userID = '123'' OR ''1'' = ''1'</code><p>
// This clause only returns those records where the userID column contains the literal value that 
// the user typed: <br>
// &nbsp;&nbsp;<code>123' OR '1' = '1</code>
// <p>
// Further, custom queries can be protected from buggy or ad-hoc client requests because the 
// query is specified on the server.  For example you could add a custom where clause, as shown
// in the above section on default clauses, to ensure that certain records are never seen by
// the client.  For instance: <p>
// <code>&lt;whereClause&gt;($defaultWhereClause) AND confidential = '0'&lt;/whereClause&gt;</code>.
// <p>
// <h4>Column case-sensitivity issues</h4>
// Different database products have different rules concerning case-sensitivity in column 
// names.  Consider the following query:
// <br><br><code>&nbsp;&nbsp;SELECT orderNumber FROM Order</code>
// <ul>
// <li>MySQL and Microsoft SQL Server are not case-sensitive with regard to column names, so 
// this query will work whether the column is called "orderNumber" or "ORDERNUMBER" or any 
// other variation.</li>
// <li>Oracle, HSQLDB and DB2 default to upper-case column names.  Therefore, this query will
// fail if the column is actually called "orderNumber"; it will only work if the underlying
// column name is "ORDERNUMBER"</li>
// <li>PostgreSQL and Informix default to lower-case column names, so this query will fail 
// unless the underlying column name is actually "ordernumber"</li>
// </ul>
// Note that these differences only apply in a practical sense if the underlying database 
// table was created with quoted column names.  If you create your tables without quoting 
// column names, the database creates the columns using its own preferred defaults, which 
// is what it will also use when it encounters an unquoted column name in a query.  Behind 
// the scenes, the differences are still there - your column will be called "ORDERNUMBER" 
// on Oracle and "ordernumber" on PostgreSQL - but that wouldn't be apparent unless you went 
// looking for it: the example query would work unchanged with both products, and you would 
// be able to use whatever mixture of case you like in your DataSource field names 
// (SmartClient will map DataSource field "orderNumber" to Oracle column "ORDERNUMBER" 
// transparently).  <b>This is the recommended approach.</b>
// <p>
// If you can't, or don't want to, accept the database default - if you are working with an
// existing schema, for example - then you will need to quote column names in your queries.
// Unfortunately, the way you do this also differs by database product, so quoting a column
// name correctly in one database's syntax may mean that the query cannot be ported to a
// different database without change.
// <p>
// To help with this case, we provide two extra container variables that you can use.
// <b>$fields</b> contains the names of all the fields in your DataSource, but quoted in
// accordance with the column-quoting rules of the target database.  <b>$qfields</b> also
// contains a list of field names, but in this case each one is qualified with its table
// name.<p>
// As an example of how to use <b>$fields</b> and <b>$qfields</b>, consider a DataSource with
// a field called "itemID", bound to a column also called "itemID", and a tableName property 
// of "orderItem".  Here are three ways to write a +link{OperationBinding.selectClause,selectClause} 
// for a custom SQL query that returns that field:<ul>
// <li><code>orderItem."itemID"</code>
// <li><code>orderItem.$fields.itemID</code>
// <li><code>$qfields.itemID</code>
// </ul>
// The first of these is not portable.  It will work fine in HSQL and Oracle, but will fail 
// with a syntax error in MySQL because you quote a field name with backticks in MySQL, not 
// quote marks.
// <p>
// The usages via <b>$fields</b> and <b>$qfields</b> <em>are</em> portable.  The second line, 
// when targeting Oracle, will be translated to <code>orderItem."itemID"</code>; when targeting
// MySQL, it will be translated to <code>orderItem.itemID</code>, or <code>orderItem.`itemID`</code>
// if column quoting is enabled for that database (it generally isn't required, since MySQL 
// preserves case by default).
//
// @treeLocation Client Reference/Data Binding/DataSource
// @title Custom Querying Overview
// @visibility customSQL
//<
        


//> @attr operationBinding.selectClause (String : null : [IR])
// <b>This feature is available with Power or better licenses only.</b> See 
// <a href=http://smartclient.com/product>smartclient.com/product</a> for details.
// <p>
// For a dataSource of +link{DataSource.serverType,serverType} "sql", this 
// property can be specified on an operationBinding to provide the server with a bespoke
// SELECT clause to use when constructing the SQL query to perform this operation.  The
// property should be a comma-separated list of column names and/or expressions, and you 
// can refer to any scalar function supported by the underlying database.  The server will 
// insert the text of this property immediately after the "SELECT" token.
// <p>
// Note that if you also specify a +link{groupClause,groupClause}, you can use aggregate 
// functions such as SUM and COUNT in the selectClause.
// <p>
// This property is only applicable to operationBindings of 
// +link{OperationBinding.operationType,operationType} "fetch".
// <p>
// See the documentation for +link{customSQL} for usage examples
//
// @group customQuerying
// @see OperationBinding.customSQL
// @serverDS only
// @visibility customSQL
//<

//> @attr operationBinding.tableClause (String : null : [IR])
// <b>This feature is available with Power or better licenses only.</b> See 
// <a href=http://smartclient.com/product>smartclient.com/product</a> for details.
// <p>
// For a dataSource of +link{DataSource.serverType,serverType} "sql", this 
// property can be specified on an operationBinding to provide the server with a bespoke
// table clause to use when constructing the SQL query to perform this operation.  The
// property should be a comma-separated list of tables and views, and you can use any
// special language constructs supported by the underlying database.  The server will 
// insert the text of this property immediately after the "FROM" token.
// <p>
// See the documentation for +link{customSQL} for usage examples
//
// @group customQuerying
// @see OperationBinding.customSQL
// @serverDS only
// @visibility customSQL
//<

//> @attr operationBinding.whereClause (String : null : [IR])
// <b>This feature is available with Power or better licenses only.</b> See 
// <a href=http://smartclient.com/product>smartclient.com/product</a> for details.
// <p>
// For a dataSource of +link{DataSource.serverType,serverType} "sql", this 
// property can be specified on an operationBinding to provide the server with a bespoke
// WHERE clause to use when constructing the SQL query to perform this operation.  The
// property should be a valid expression in the syntax of the underlying database.  The 
// server will insert the text of this property immediately after the "WHERE" token.
// <p>
// You may find the SmartClient-provided <b>$criteria</b> variable of particular use with
// this property.
// <p>
// See the documentation for +link{customSQL} for usage examples
//
// @group customQuerying
// @see OperationBinding.customSQL
// @serverDS only
// @visibility customSQL
//<

//> @attr operationBinding.groupClause (String : null : [IR])
// <b>This feature is available with Power or better licenses only.</b> See 
// <a href=http://smartclient.com/product>smartclient.com/product</a> for details.
// <p>
// For a dataSource of +link{DataSource.serverType,serverType} "sql", this 
// property can be specified on an operationBinding to provide the server with a bespoke
// GROUP BY clause to use when constructing the SQL query to perform this operation.  The
// property should be a comma-separated list of column names and/or expressions, forming
// a valid GROUP BY clause in the syntax of the underlying database.  The server will 
// insert the text of this property immediately after the "GROUP BY" token.
// <p>
// Note that specifying this property enables you to use aggregate functions (such as 
// COUNT and SUM) in your +link{selectClause,selectClause}. Also note that care is required 
// when using groupClause to ensure that the selectClause contains the fields you are grouping
// by.  Failure to do this correctly will result in a runtime SQL error.
// <p>
// This property is only applicable to operationBindings of 
// +link{OperationBinding.operationType,operationType} "fetch".
// <p>
// See the documentation for +link{customSQL} for usage examples
//
// @group customQuerying
// @see OperationBinding.customSQL
// @see OperationBinding.selectClause
// @serverDS only
// @visibility customSQL
//<

//> @attr operationBinding.orderClause (String : null : [IR])
// <b>This feature is available with Power or better licenses only.</b> See 
// <a href=http://smartclient.com/product>smartclient.com/product</a> for details.
// <p>
// For a dataSource of +link{DataSource.serverType,serverType} "sql", this 
// property can be specified on an operationBinding to provide the server with a bespoke
// ORDER BY clause to use when constructing the SQL query to perform this operation.  The
// property should be a comma-separated list of column names and/or expressions, forming
// a valid ORDER BY clause in the syntax of the underlying database.  The server will 
// insert the text of this property immediately after the "ORDER BY" token.
// <p>
// This property is only applicable to operationBindings of 
// +link{OperationBinding.operationType,operationType} "fetch".
// <p>
// See the documentation for +link{customSQL} for usage examples
//
// @group customQuerying
// @see OperationBinding.customSQL
// @serverDS only
// @visibility customSQL
//<

//> @attr operationBinding.valuesClause (String : null : [IR])
// <b>This feature is available with Power or better licenses only.</b> See 
// <a href=http://smartclient.com/product>smartclient.com/product</a> for details.
// <p>
// For a dataSource of +link{DataSource.serverType,serverType} "sql", this 
// property can be specified on an operationBinding to provide the server with a bespoke
// set of values to add or update,  for use when constructing the SQL query to perform this 
// operation.  The property should be one of the following, depending on the 
// +link{operationType,operationType}:
// <p>
// For "add" operations, the syntax that would be valid for an INSERT INTO query: a 
// comma-separated list of column names enclosed in parentheses, followed by a comma-separated
// list of new values, enclosed in parentheses and preceded by the token "VALUES".  For example:
// <p>
// <code>&lt;valuesClause&gt;(name, age) VALUES("Jane Doe", 48)&lt;/valuesClause&gt;</code>
// <p>
// For "update" operations, the syntax that would be valid for an UPDATE query: a 
// comma-separated list of expressions equating a column name to its new value.  For example:
// <p>
// <code>&lt;valuesClause&gt;name="Jane Doe", age=48&lt;/valuesClause&gt;</code>
// <p>
// You may find the SmartClient-provided <b>$values</b> variable of particular use with
// this property.
// <p>
// See the documentation for +link{customSQL} for usage examples
//
// @group customQuerying
// @see OperationBinding.customSQL
// @serverDS only
// @visibility customSQL
//<

//> @attr operationBinding.customSQL (String : null : [IR])
// <b>This feature is available with Power or better licenses only.</b> See 
// <a href=http://smartclient.com/product>smartclient.com/product</a> for details.
// <p>
// For a dataSource of +link{DataSource.serverType,serverType} "sql" or "hibernate", this 
// property can be specified on an operationBinding to indicate that the server should run 
// user-specified SQL, rather than the query it would normally generate to satisfy a 
// dataSource operation.  This property allows you to provide a fully-customized query; an
// alternative is to provide custom "pieces" to the query generator, via properties such as
// +link{whereClause,whereClause} and +link{valuesClause,valuesClause}.  See the 
// +link{group:customQuerying} for more details.
// <p>
// For a dataSource of type "sql", the SmartClient server generates a number of useful
// +link{type:DefaultQueryClause, query "pieces"}, 
// and makes them available to your custom SQL code via the Velocity templating 
// language (note that this is not available for "hibernate" dataSources).
// <p>
// We also make the template variables <b>$criteria</b> and <b>$values</b> available, to give
// you direct access to the supplied criteria, and to the new field values for update and add
// operations.  These variables are available to both "sql" and "hibernate" dataSources.
// <p>
// Note that you should use this feature with care. In particular, writing customSQL code
// that makes use of a particular database engine's features or syntax will make your 
// application less portable.  
// <p>
// See +link{group:customQuerying} for an overview of writing custom queries and clauses.
// <p>
// <h3>Examples</h3>
// An example using the SmartClient-supplied query pieces.  This custom query will give 
// exactly the same result as the SmartClient-generated query:
// <p>
// <code>
// &lt;operationBinding operationId="customFetch" operationType="fetch"&gt;<br>
// &nbsp;&nbsp;&lt;customSQL&gt;<br>
// &nbsp;&nbsp;&nbsp;&nbsp;SELECT $defaultSelectClause FROM $defaultTableClause
// WHERE $defaultWhereClause ORDER BY $defaultOrderClause<br>
// &nbsp;&nbsp;&lt;/customSQL&gt;<br>
// &lt;/operationBinding&gt;<br>
// </code>
// <p>
// An example using the SmartClient-supplied <b>$criteria</b> template variable:
// <p>
// <code>
// &lt;operationBinding operationId="customFetch" operationType="fetch"&gt;<br>
// &nbsp;&nbsp;&lt;customSQL&gt;<br>
// &nbsp;&nbsp;&nbsp;&nbsp;SELECT foo, bar, baz FROM MyTable WHERE bar &gt; $criteria.someValue<br>
// &nbsp;&nbsp;&lt;/customSQL&gt;<br>
// &lt;/operationBinding&gt;<br>
// </code>
// <p>
// An update example:
// <p>
// <code>
// &lt;operationBinding operationId="myUpdateOp" operationType="update"&gt;<br>
// &nbsp;&nbsp;&lt;customSQL&gt;<br>
// &nbsp;&nbsp;&nbsp;&nbsp;UPDATE $defaultTableClause SET $defaultValuesClause WHERE bar 
// &lt;= $criteria.someValue<br>
// &nbsp;&nbsp;&lt;/customSQL&gt;<br>
// &lt;/operationBinding&gt;<br>
// </code>
//
// @group customQuerying
// @see OperationBinding.customHQL
// @see DataSourceField.customSQL
// @serverDS only
// @visibility customSQL
//<



//> @attr operationBinding.customHQL (String : null : [IR])
// <b>This feature is available with Power or better licenses only.</b> See 
// <a href=http://smartclient.com/product>smartclient.com/product</a> for details.
// <p>
// For a dataSource of +link{DataSource.serverType,serverType} "hibernate", this 
// property can be specified on an operationBinding to indicate that the server should run 
// user-specified HQL (Hibernate Query Language), rather than the Hibernate criteria query or 
// <code>saveOrUpdate</code> call it would normally generate to satisfy a dataSource operation
// via Hibernate. 
// <p>
// Note that inserting new records via HQL is often impractical, due to intentional restrictions
// in the language (it is only possible to perform an insert expressed in terms of a SELECT; the
// "VALUES" construct commonly used when inserting new rows singly is not supported).  If you are 
// intending to use customHQL, we recommend that you avoid doing so for +link{operationBinding}s 
// with +link{OperationBinding.operationType,operationType} "add", unless you have a special 
// requirement such as a bulk insert; if you need custom queries to perform inserts 
// on "hibernate" dataSources, we recommend you use +link{OperationBinding.customSQL,customSQL},
// which is valid for "hibernate" DataSources as well as "sql" dataSources.
// <p>
// For other operations on "hibernate" dataSources, however, HQL has the advantage of 
// being more portable across different database engines than is plain SQL.
// <p>
// Note that using customHQL affects paging implementation. If you use it, full data set is fetched 
// from Hibernate and records that aren't in the requested range are dropped at the server side.
//
// @group customQuerying
// @see OperationBinding.customSQL
// @see DataSourceField.customSQL
// @serverDS only
// @visibility customSQL
//<

//> @attr operationBinding.skipRowCount (boolean : null : [IR])
// A SQLDataSource will normally issue two queries for a "fetch" operation when paging is
// enabled: one to determine the total rows available (the "row count query"), and one to fetch
// the specific range of rows requested.
// <P>
// For very large datasets the row count query can be expensive, and it can make more sense to
// use the "progressive loading" strategy discussed in the +link{ResultSet} docs.
// <P>
// Setting skipRowCount="true" will avoid the "row count query", but as a consequence
// +link{dsResponse.totalRows} will be set to match the requested +link{dsRequest.endRow} since
// the totalRows is unknown.  To implement the progressive loading pattern, use a +link{DMI} to
// to set totalRows to be larger than the requested endRow by the number of rows you would like
// the user to be able to scroll ahead in one jump.
//
// @serverDS only
// @visibility customSQL
//<

//> @attr operationBinding.customFields (String or Array: null : [IR])
// Indicates that the listed fields should be included in the default
// +link{selectClause,selectClause} and +link{selectClause,whereClause} generated for this 
// operationBinding, even if they are marked +link{listGridField.customSQL,customSQL="true"}.
// <P>
// If you need to apply different sets of overrides for the <code>selectClause</code> and the
// <code>whereClause</code>, use +link{customValueFields,customValueFields} and/or 
// +link{customCriteriaFields,customCriteriaFields} instead.  If you specify both
// <code>customFields</code> and <code>customCriteriaFields</code> or 
// <code>customValueFields</code>, the more specific variant wins.  If you specify both 
// <code>customFields</code> and +link{excludeCriteriaFields,excludeCriteriaFields}, 
// <code>customFields</code> wins (this is another use case when you may wish to use 
// <code>customValueFields</code> instead)
// <p>
// You can specify this property as a comma-separated list (eg, "foo, bar, baz") or by just
// repeating the &lt;customFields&gt; tag multiple times with one field each.
// <p>
// This property is only applicable to DataSources of +link{dataSource.serverType,"sql"}.
//
// @group customQuerying
// @see OperationBinding.customValueFields
// @see OperationBinding.customCriteriaFields
// @see OperationBinding.excludeCriteriaFields
// @serverDS only
// @visibility customSQL
//<        

//> @attr operationBinding.customValueFields (String or Array: null : [IR])
// Indicates that the listed fields should be included in the default
// +link{selectClause,selectClause} generated for this operationBinding, even if they are marked 
// +link{listGridField.customSQL,customSQL="true"}.
// <P>
// You can specify this property as a comma-separated list (eg, "foo, bar, baz") or by just
// repeating the &lt;customValueFields&gt; tag multiple times with one field each.
// <p>
// This property is only applicable to DataSources of +link{dataSource.serverType,"sql"}.
//
// @group customQuerying
// @see OperationBinding.customFields
// @see OperationBinding.customCriteriaFields
// @serverDS only
// @visibility customSQL
//<        

//> @attr operationBinding.customCriteriaFields (String or Array : null : [IR])
// Indicates that the listed fields should be included in the default
// +link{whereClause,whereClause} generated for this operationBinding, even if they are marked 
// +link{dataSourceField.customSQL,customSQL="true"}.
// <P>
// You can specify this property as a comma-separated list (eg, "foo, bar, baz") or by just
// repeating the &lt;customCriteriaFields&gt; tag multiple times with one field each.
// <p>
// This property is only applicable to DataSources of +link{dataSource.serverType,"sql"}.
//
// @group customQuerying
// @see OperationBinding.customFields
// @see OperationBinding.customValueFields
// @see OperationBinding.excludeCriteriaFields
// @serverDS only
// @visibility customSQL
//<        

//> @attr operationBinding.excludeCriteriaFields (String or Array: null : [IR])
// Indicates that the listed fields should be excluded from the default
// +link{whereClause,whereClause} generated for this operationBinding.  
// <P>
// This enables you to use these fields in a +link{group:customQuerying,custom query} while
// still allowing the $defaultWhereClause to be generated for all other fields.  For example,
// you might take a particular field and apply it in the WHERE clause of a subquery.
// <p>
// You can specify this property as a comma-separated list (eg, "foo, bar, baz") or by just
// repeating the &lt;customCriteriaFields&gt; tag multiple times with one field each.
// Note that if a field is included in both excludeCriteriaFields and
// +link{operationBinding.customCriteriaFields,customCriteriaFields}, customCriteriaFields
// wins.
// <p>
// This property is only applicable to DataSources of +link{dataSource.serverType,"sql"}.
//
// @group customQuerying
// @see OperationBinding.customCriteriaFields
// @serverDS only
// @visibility customSQL
//<        
        
//> @attr operationBinding.useForCacheSync (boolean : true : [IR])
// For an operationBinding of +link{operationType,operationType} "fetch" which specifies no
// +link{operationId,operationId}, this property determines whether the operationBinding 
// should be used for cache synchronization purposes (ie, to retrieve the record most recently
// added or changed).  This property has no effect on an operationBinding that specifies an 
// operationId - see +link{cacheSyncOperation}.
// <p>
// In order to work correctly with SmartClient's cache synchronization system, an 
// operationBinding marked useForCacheSync should have the following properties:
// <ul>
// <li>Able to complete its retrieval using no context other than the values of the primary key
// fields declared in the dataSource (these will be provided in the $criteria object passed to 
// the operation)</li>
// <li>Returns the entire record, including any values that may require joins to other tables
// or other complexities</li>
// </ul>
// This property is only applicable to DataSources of type "sql".
//
// @group customQuerying
// @see OperationBinding.cacheSyncOperation
// @serverDS only
// @visibility customSQL
//<        

//> @attr operationBinding.cacheSyncOperation (String : null : [IR])
// For an operationBinding of +link{operationType,operationType} "add" or "update", this 
// property is the +link{operationId,operationId} of a "fetch" operationBinding to use 
// for cache synchronization purposes (ie, to fetch the row most recently added or changed).
// This property, along with +link{useForCacheSync,useForCacheSync} and 
// +link{canSyncCache,canSyncCache} is provided so that you can use custom database operations
// without sacrificing the benefits of SmartClient's automatic cache synchronization.
// <p>
// This property is only applicable to DataSources of type "sql".
//
// @group customQuerying
// @see OperationBinding.useForCacheSync
// @see OperationBinding.canSyncCache
// @serverDS only
// @visibility customSQL
//<        

//> @attr operationBinding.relaxSecurityForCacheSync (Boolean : null : [IR])
// For a "fetch" operationBinding which is to be used as the cache-sync operation for one or 
// more "add" or "update" operations, set this flag to prevent the server from applying any 
// +link{requiresRole,declarative security} rules when this operation is run <em>for cache-sync
// purposes</em>.  If the operation is run as a normal client request, all normal security 
// processing applies regardless of this flag.
// <p>
// This property can help to avoid the need for coded workarounds in some security edge cases.
// For example, consider a secure application that only allows authenticated users; this
// application would naturally specify 
// +link{dataSource.requiresAuthentication,requiresAuthentication}: <code>true</code> at the 
// DataSource level, to enforce that requirement.  However, this causes a problem when a new 
// user registers: we can provide a special "add" operation just for registering, that does
// not require authentication, but that is going to cause a cache-sync fetch that <b>does</b>
// require authentication, and that is going to fail because the user is not yet authenticated.
// <p>
// We could provide a special cache-sync fetch operation that does not require authentication,
// and hook it up to the special registration operation via the 
// +link{cacheSyncOperation,cacheSyncOperation} setting.  However, that operation can also be 
// executed from ordinary client code, leading to a situation where malicious code could
// obtain user details without being authenticated.
// <p>
// For this use case, the best solution is to create a special cache-sync fetch operation as 
// described, but to specify normal security settings.  Then, specify 
// <code>relaxSecurityForCacheSync</code> on that operationBinding to make the system skip 
// security checks only when the operation is actually executing as part of a cache-sync fetch.
//
// @group customQuerying
// @see OperationBinding.cacheSyncOperation
// @see OperationBinding.useForCacheSync
// @see OperationBinding.canSyncCache
// @serverDS only
// @visibility internal
//<        

//> @attr operationBinding.canSyncCache (Boolean : null : [IR])
// For an operation of type "add" or "update", a SQLDataSource will normally obtain
// data to return to the client by performing the "cacheSyncOperation": a SELECT statement that
// retrieves the modified record by primary key.  This accommodates sequence columns, columns
// with default values, database triggers and other database features that may modify data
// after insertion or update.
// <P>
// Certain major SQL customizations can prevent the SQLDataSource from authoritatively
// determining the primary key used in the SQL statement, such that re-selecting the saved
// record may fail.  By default, when <code>canSyncCache</code> has not been explicitly
// set, in the following cases it is assumed that the normal cacheSyncOperation cannot be used:
// <ul>
// <li> &lt;customSQL&gt; has been used to define an entirely custom query
// <li> a custom &lt;whereClause&gt; has been defined for an "update" or "remove" operation
// <li> a custom &lt;valuesClause&gt; has been defined for an "add" operation
// </ul>
// If any of these cases apply or if <code>canSyncCache</code> has been set false, the server
// will skip the cacheSyncOperation and return a DSResponse where
// +link{dsResponse.invalidateCache} has been set to true to notify client-side components that
// they may need to refresh their entire cache.
// <P>
// Alternatively, if the default re-selection behavior will not work but a customized SQL
// query would work, you can define that SQL operation as another operationBinding and use
// +link{operationBinding.cacheSyncOperation} to declare that it should be used.  Setting
// <code>cacheSyncOperation</code> implicitly sets <code>canCacheSync</code> to true.
// <P>
// This property is only applicable to DataSources of type "sql".
//
// @group customQuerying
// @see OperationBinding.useForCacheSync
// @see OperationBinding.cacheSyncOperation
// @serverDS only
// @visibility customSQL
//<        

//> @attr operationBinding.sqlType (SQLType : null : [IR])
// For dataSources of +link{DataSource.serverType,serverType} "sql" and "hibernate" only, this 
// property determines whether "custom" operations have their custom SQL or HQL sent to the  
// underlying database via a JDBC <code>executeQuery()</code> or a JDBC 
// <code>executeUpdate()</code>.  The default value of null means the same as "query", so you
// only need to use this property when your custom SQL or HQL updates data.
//
// @group customQuerying
// @serverDS only
// @visibility customSQL
//<

//> @type SQLType
// The types of custom query that can be handled by SmartClient's built-in "sql" and 
// "hibernate" DataSources.  Note, only applies to +link{operationBinding.operationType}
// "custom".
//
// @value "query"   The custom SQL or HQL is read-only
// @value "update"  The custom SQL or HQL updates data
//
// @group customQuerying
// @visibility customSQL
//<
        

// Miscellaneous
// --------------------------------------------------------------------------------------------

//> @attr operationBinding.allowMultiUpdate (boolean : null : [IR])
// Ordinarily, "update" and "remove" operations are only allowed for +link{class:DataSource}s
// that have a +link{attr:DataSourceField.primaryKey,primaryKey}, and all primary key values
// are present in the request.  This is because an update of a DataSource with no primary key, 
// or an update request that has missing primary key values, cannot be guaranteed to affect 
// only one record.
// <p>
// Setting this property on an operationBinding circumvents this restriction for that operation
// only.
// <p>
// <b>Warning:</b> Be aware that this is a potentially dangerous setting and should be used
// with care.  With this flag set, you have no guarantee that an update will not change or 
// remove every row in a table.
//
// @serverDS only
// @visibility external
//<

//> @attr operationBinding.qualifyColumnNames (boolean : true : [IR])
// Specifies, for this specific operationBinding, whether to qualify column names with table
// names in any SQL we generate.  Overrides the +link{DataSource.qualifyColumnNames} property.
// Only applicable to dataSources of +link{DataSource.serverType,serverType} "sql".
//
// @serverDS only
// @visibility external
//<

//> @attr operationBinding.invalidateCache (boolean : null : [IR])
// If set, every invocation of this operationBinding will invalidate the local cache, forcing
// a server visit to refresh the data.
//
// @serverDS allowed
// @visibility external
//<

//> @attr operationBinding.outputs (String: null : [IR])
// Specifies, for this operationBinding only, the list of field names that should be returned
// to the client.  Typically this will be a subset of the +link{DataSource.fields}, but note 
// that this is not a requirement; <code>outputs</code> can include fields that are not defined
// in the DataSource's field list.  In this case, the server will return extra fields even if 
// +link{dataSource.dropExtraFields} is true.
// <p>
// You specify this property as a string containing a comma-separated list of field names 
// (eg, "foo, bar, baz")
// 
// @serverDS only
// @visibility external
//<

	
//> @attr operationBinding.sqlPaging (SQLPagingStrategy : null : IRW)
// The paging strategy to use for this specific OperationBinding.  If this property is not
// set, we fall back to the +link{DataSource.sqlPaging} value, and the defaults described in
// the documentation for that property.
//
// @see dataSource.sqlPaging
// @serverDS only
// @visibility external
//<

//> @attr operationBinding.sqlUsePagingHint (boolean : null : IR)
// If explicitly set true or false, forces the use of a "hint" in the SQL we generate for paged 
// queries on or off as appropriate.  If not set, defaults to the 
// +link{dataSource.sqlUsePagingHints} value.  Note this property is only applicable to 
// +link{dataSource.serverType,SQL} DataSources, only when a 
// +link{dataSource.sqlPaging,paging strategy} of "sqlLimit" is in force, and it only has an 
// effect for those specific database products where we employ a native hint in the generated 
// SQL in an attempt to improve performance.
//
// @requiresModules SCServer
// @see dataSource.sqlUsePagingHint
// @group sqlPaging
// @serverDS only
// @visibility external
//<

	
//> @attr operationBinding.useSubselectForRowCount (boolean : null : IRW)
// Whether to use the subselect technique (see +link{DataSource.useSubselectForRowCount} for
// details) to derive a rowcount query for this operation.  If this property is not
// set, we fall back to the <code>useSubselectForRowCount</code> setting on the DataSource, 
// and the defaults described in the documentation for that property.
//
// @see dataSource.useSubselectForRowCount
// @see operationBinding.customSQL
// @serverDS only
// @visibility external
//<


	
//> @attr operationBinding.progressiveLoading (boolean : null : IRW)
// Sets +link{DataSource.progressiveLoading,progressive loading mode} for this particular 
// operation, overriding the DataSource-level setting.  Note that this setting applies only
// to fetch operations - it has no effect if specified on any other kind of operation.
//
// @see dataSource.progressiveLoading
// @serverDS only
// @group progressiveLoading
// @visibility external
//<

	
//> @attr operationBinding.allowAdvancedCriteria (boolean : null : IRWA)
// This property indicates whether this operation supports AdvancedCriteria. This setting 
// overrides +link{dataSource.allowAdvancedCriteria} for this operation only. See
// +link{dataSource.supportsAdvancedCriteria()} for further information.
// <p>
// <b>NOTE:</b> If you specify this property in a DataSource descriptor 
// (<code>.ds.xml</code> file), it is enforced on the server.  This means that if you run 
// a request containing AdvancedCriteria against an OperationBinding that advertises itself
// as <code>allowAdvancedCriteria:false</code>, it will be rejected.
// @see dataSource.allowAdvancedCriteria
// @serverDS allowed
// @visibility external
//<



//> @object DSRequestModifier
// <b>Some elements of this feature are only available with Power or better licenses.</b> See 
// <a href=http://smartclient.com/product>smartclient.com/product</a> for details.
// <p>
// An object that modifies a dsRequest based on several available 
// values, including preceding responses from the same queue. You provide a list of 
// DSRequestModifiers as the +link{operationBinding.criteria} or +link{operationBinding.values}
// of an +link{OperationBinding}.
// <P>
// A DSRequestModifier consists of a +link{dsRequestModifier.fieldName,fieldName}, usually a
// +link{dsRequestModifier.value,value} and possibly an +link{dsRequestModifier.operator,operator}
// and +link{dsRequestModifier.start,start} and/or +link{dsRequestModifier.end,end} values 
// (applicable to advanced criteria only).  The value, start and end settings can be static, or - 
// with Power or better licenses - they can be expressions in the Velocity template language, 
// which will be resolved at runtime, immediately before the DSRequest is executed. 
//
// @treeLocation Client Reference/Data Binding/DataSource
// @serverDS only
// @visibility external
//<

//> @attr dsRequestModifier.fieldName (String : null : [IR])
// The name of the field to add or replace on the DSRequest - whether this appears in the DSRequest's
// values or criteria depends on whether this is part of a +link{operationBinding.values} or
// +link{operationBinding.criteria} attribute.
// 
// @serverDS only
// @visibility external
//<

//> @attr dsRequestModifier.operator (String : null : [IR])
// The name of the operator to apply when constructing criteria.  This property only applies 
// to criteria; it is ignored if specified as part of a +link{operationBinding.values} 
// attribute.
// 
// @serverDS only
// @visibility external
//<

//> @attr dsRequestModifier.value (String : null : [IR])
// The value to assign to the field named by +link{fieldName}.  This value can be static, and 
// for Pro licenses that is the only option.  With Power and better licenses, this value 
// can be an expression in the Velocity template language.  In this latter case, there are a 
// number of context variables available to you:
// <ul><li><b>dsRequest</b>. The DSRequest object being processed</li>
// <li><b>servletRequest</b>. The HttpServletRequest object representing the queue of requests
// sent from the client</li>
// <li><b>responseData</b>. The data member of a DSResponse object associated with an earlier 
// DSRequest in the same queue</li></ul>
// (Note that the standard Velocity context variables provided by SmartClient Server are also 
// available to you - see +link{group:velocitySupport}).
// <p>
// The most interesting of these is the last. An example of when this might be useful would be a
// queued batch of "add" operations for an order header and its details - the detail additions need
// to know the unique primary key that was assigned to the order.  This value will be in the 
// response to the DSRequest that added the order header.  See this example:
// +explorerExample{queuedAdd}.
// <p>
// You can obtain the responseData of prior responses using the following Velocity syntax:<p>
// <code>$responseData.first.myField</code> is the myField property of the first response in the queue<br>
// <code>$responseData.first('order').myField</code> is the myField property of the first response to an 
// operation (any operation) on the "order" DataSource<br>
// <code>$responseData.first('order', 'add').myField</code> is the myField property of the first 
// response to an "add" operation on the "order" DataSource<p>
// All of these syntactic variations are also available on the <code>$responseData.last</code> object - 
// "last" here meaning the most recent response matching the DataSource and operation type (if 
// applicable).  Please see the server-side Javadoc for the
// <code>com.isomorphic.velocity.ResponseDataHandler</code> class.
// 
// @see group:velocitySupport
// @serverDS only
// @visibility external
//<

//> @attr dsRequestModifier.start (String : null : [IR])
// The value to use for the start of a range.  This property only applies to criteria, and it 
// only applies to operators of type "rangeCheck" - for example, the "between" operator.
// It is ignored if specified as part of a +link{operationBinding.values} attribute, or for 
// an inapplicable operator type.
// <p>
// The same rules apply to this attribute as apply to +link{dsRequestModifier.value,value}, so
// you can use Velocity expressions if you have a Power or better license.
// 
// @serverDS only
// @visibility external
//<

//> @attr dsRequestModifier.end (String : null : [IR])
// The value to use for the end of a range.  This property only applies to criteria, and it 
// only applies to operators of type "rangeCheck" - for example, the "between" operator.
// It is ignored if specified as part of a +link{operationBinding.values} attribute, or for 
// an inapplicable operator type.
// <p>
// The same rules apply to this attribute as apply to +link{dsRequestModifier.value,value}, so
// you can use Velocity expressions if you have a Power or better license.
// 
// @serverDS only
// @visibility external
//<

//> @attr operationBinding.criteria (List of DSRequestModifier: null : [IR])
// <b>Elements of this feature are only available with Power or better licenses.</b> See 
// <a href=http://smartclient.com/product>smartclient.com/product</a> for details.
// <p>
// A list of +link{DSRequestModifier}s that will be used to modify the criteria of each 
// +link{DSRequest} that uses this operationBinding.  Note that the criteria elements are 
// applied to DSRequest criteria as follows:
// <ul>
// <li><b>Simple criteria:</b> The field and value are just applied as an extra key/value 
// pair in the criteria map, as long as the +link{dsRequestModifier.operator,operator} 
// attribute is left unset, or is set to "equals".  For any other setting of 
// <code>operator</code>, the criteria is first converted to the equivalent AdvancedCriteria 
// and then processed as described below</li>
// <li><b>AdvancedCriteria:</b> If the topmost operator is "and", we add the new criterion 
// as an additional criterion directly in the existing list.  Otherwise, we create a new 
// top-level AdvancedCriteria with an operator of "and".  This is then set to have two 
// elements in its criteria: the previous top-level criteria and the new criterion.</li>
// </ul>
// The effect of this is to apply any criteria specifed here as additional constraints on top
// of what the user has specified, and of course, the user is unable to affect this.  Thus, 
// this is a suitable and convenient place to enforce rules such as "Users can only ever see
// their own records".
//
// @see operationBinding.values
// @serverDS only
// @visibility external
//<        

//> @attr operationBinding.values (List of DSRequestModifier: null : [IR])
// <b>Elements of this feature are only available with Power or better licenses.</b> See 
// <a href=http://smartclient.com/product>smartclient.com/product</a> for details.
// <p>
// A list of +link{DSRequestModifier}s that will be used to modify the values object of each 
// +link{DSRequest} that uses this operationBinding.
//
// @see operationBinding.criteria
// @serverDS only
// @visibility external
//< 

//> @type XPathExpression
// A standard XPath expression as a string. To learn about XPath, try the following search:
// <a href="http://www.google.com/search?q=xpath+tutorial" target="_blank"
// >http://www.google.com/search?q=xpath+tutorial</a>
// @visibility external
//<    

//> @type VelocityExpression
// An expression in the <a href=http://velocity.apache.org/engine/releases/velocity-1.5/user-guide.html>
// Velocity Template Language</a> (VTL).  For more information on SmartClient's Velocity support, 
// see +link{group:velocitySupport, Velocity support}.
// <p>
// Note that a <code>VelocityExpression</code> must often evaluate to a particular type of value 
// to be useful.  For example, +link{DataSource.requires} must evaluate to true or false 
// (Boolean objects or strings containing those two words), and +link{Mail.messageData} must
// evaluate to a Java <code>Map</code> object, or a Java <code>List</code> containing only 
// <code>Map</code>s.
//
// @group velocitySupport
// @treeLocation Client Reference/Data Binding/DataSource
// @visibility external
//<

// Declarative security
// -----------------------------------------------------------------------------------------
//> @attr operationBinding.requiresAuthentication (boolean : null : IR)
// Whether a user must be authenticated in order to access this operation.  For details of 
// what is meant by "authenticated", see +link{DataSource.requiresAuthentication}.
// <P>
// To protect access to an entire operationType (eg, all "fetch" operations), declare an
// operationBinding with <code>requiresAuthentication="true"</code>, +link{operationType} set
// to the operationType to be protected, but no +link{operationId}.  This will then
// prevent access to the "fetch" operationType unless another +link{operationBinding}
// declares requiresAuthentication="false" with a specific
// +link{operationBinding.operationId,operationId}.
// 
// @requiresModules SCServer
// @serverDS only
// @group auth
// @visibility external
//<

//> @attr operationBinding.requiresRole (String : null : IR)
// Comma-separated list of user roles that are allowed to invoke the operation described by
// this operationBinding. If the current user has any of the roles listed, they can
// invoke the operation. Also note that <code>authentication.superuserRole</code> can be 
// specified in the +link{server_properties,server.properties} file. If set this denotes a "super user" 
// role - any user with that role will have access to all operations, regardless of the
// "requiresRole" settings for the operation.
// <P>
// Whether the current user has a given role is determined by calling the standard Java
// servlets method <code>httpServletRequest.isUserInRole()</code>, hence works with both simple
// J2EE security (realms and form-based authentication) and JAAS (Java Authentication &
// Authorization Service).
// <P>
// If you wish to use a role-based security scheme that does not make use of the servlet API's 
// standards, SmartClient Server also implements the <code>setAuthenticated</code> and
// <code>setUserRoles</code> methods on <code>RPCManager</code>. 
// You can use this API to tell SmartClient that all the requests in the queue currently 
// being processed are associated with a user who has the roles you supply; in this case,
// SmartClient will not attempt to resolve the user's roles via 
// <code>httpServletRequest.isUserInRole()</code>. When taking this approach, the 
// <code>rpcManager.setUserRoles()</code> method should be called on the server for 
// each transaction received from the client. We recommend doing this by overriding the 
// special IDACall servlet and checking server side state to determine the current user's 
// roles, calling the API, and then calling <code>handleDSRequest()</code> or
// <code>handleRPCRequest()</code> directly to handle the request(s) passed in.<br>
// Here's an example of this approach which assumes the current user's roles has been
// set directly on the HttpSession object as a comma-separated-string attribute "roles":
// <pre><code>
//  public class SecureIDACall extends IDACall {
//		
//      public void processRequest(HttpServletRequest request,
//              HttpServletResponse response)
//       throws ServletException, IOException
//      {
//          HttpSession session = request.getSession();
//          Object roles = session == null ? null : session.getAttribute("roles");
// 
//          if (roles != null) {
//              try {
//                  RequestContext context = RequestContext.instance(this, request, response);   
//                  RPCManager rpc = new RPCManager(request, response);
//                  rpc.setAuthenticated(true);
//                  rpc.setUserRoles((String) roles); 
//                  
//                  // call processRPCTransaction() to iterate through all RPCRequests and
//                  // DSRequests and execute them
//                  processRPCTransaction(rpc, context);
// 
//              } catch (Throwable e) {
//                  handleError(response, e);
//              }
//          } else {
//              super.processRequest(request, response);
//          } 
//      }
//  }
// </code></pre>
// <P>
// If there is an operationBinding declared for a given operationType which does not have an
// +link{operationId}, that is, it is the default operationBinding for the type, then any other
// operationBinding of the same type is assumed to have the same setting for
// <code>requiresRole</code> as the default operationBinding for the operationType.  For
// example, given these declarations:
// <pre>
//     &lt;operationBinding operationType="fetch" requiresRole="manager"&gt;
//           ... settings ...
//      &lt;/operationBinding&gt;
//     &lt;operationBinding operationType="fetch" operationId="fetchWithExtraFields"&gt;
//           ... settings ...
//      &lt;/operationBinding&gt;
// </pre>
// The second operationBinding requires the "manager" role even though there is no explicit
// <code>requiresRole</code> declaration.  To prevent the "manager" role being required by the
// second operationBinding, add <code>requireRole=""</code>.
// <P>
// Note that if +link{dataSource.requiresRole} is set, all operations on the DataSource require
// the roles set for the DataSource as a whole, even if they declare individual
// <code>requiresRole</code> attributes.
// <P>
// This property is valid only for a server-side DataSource when using the SmartClient Server.
//
// @serverDS only
// @requiresModules SCServer
// @group auth
// @visibility external
//<

//> @attr operationBinding.requires (VelocityExpression : null : IR)
// Indicates that the specified +link{VelocityExpression} must be true for a user to access
// this operationBinding.
// <P>
// As with +link{operationBinding.requiresRole}, if there an operationBinding that is the
// default operationBinding for the operationType, its <code>requires</code> expression is
// assumed to apply to all other operationBindings of the same type unless they explicitly set
// <code>requires=""</code>
// <P>
// +link{DataSource.requires}, if specified, applies before
// <code>operationBinding.requires</code> is evaluated.  In this case, both <code>requires</code>
// expressions must be true for the request to be accepted.
//
// @serverDS only
// @group auth
// @visibility external
//<


//> @attr operationBinding.creatorOverrides (boolean : null : IR)
// Indicates that field-level declarative security rules are waived for rows that were 
// created by the current user, as described in the discussion of 
// +link{DataSource.creatorOverrides,dataSource.creatorOverrides}.  This setting overrides 
// <code>dataSource.creatorOverrides</code>, for this operation only.
//
// @requiresModules SCServer
// @see dataSourceField.editRequires
// @see dataSourceField.viewRequires
// @see dataSource.creatorOverrides
// @group fieldLevelAuth
// @serverDS only
// @visibility external
//<


// Template-based email
// -----------------------------------------------------------------------------------------

//> @attr operationBinding.mail (Mail : null : IR)
// Definition of an email message that will be sent as an after-effect of selecting or updating
// data.
// <p>
// Note that if a fixed number of different messages need to be sent, multiple 
// <code>&lt;mail&gt;</code> tags may be specified.  For example, one mail could be sent to an 
// admin address, and a different message to every member of a user group.
// 
// @visibility external
// @serverDS only
// @group mail
//<

//> @class Mail
// A <code>Mail</code> object describes an email that will be sent automatically by the 
// SmartClient Server as part of a normal +link{class:DataSource} operation.  Mail definitions
// are part of an +link{class:operationBinding}, and they can only be specified in a 
// DataSource's XML definition.  Despite the declarative nature of mail definitions, templating
// support via Velocity makes it very flexible; it is very easy to include values from records
// just fetched from permanent storage by this operation, for example.  See 
// +link{group:velocitySupport} for an overview of SmartClient Server's pervasive support for 
// templated declarative operations using Velocity. 
// <P>
// The default context for the message is the dsResponse.data; that is, $foo in the
// message refers to the field "foo" in the DataSource response (the equivalent of
// calling dsResponse.getFieldValue("foo") in Java).  You also have access to the normal
// Velocity variables described in the overview linked to above.
// <P>
// If multiple records appear in the dsResponse, one message will be sent per record
// in the response unless "multiple" is set to false.  This allows a set of
// recipients to be selected by a "fetch" operationBinding and messages sent to each.
// This also means that if no data is selected by a fetch or affected by an update,
// no mail is sent.
// <P>
// If you have the Transaction Chaining feature, all of the variables normally
// accessible in other Velocity contexts are available in the message template as
// well, for example, $responseData.last('order').orderId.  See 
// +link{dsRequestModifier.value,transaction chaining} for more details.
// <p>
// You can provide the body of the email directly in the +link{mail.messageTemplate,messageTemplate}
// tag, or you can provide a filename in the +link{mail.templateFile,templateFile} attribute,
// which tells SmartClient to use the contents of that file as the body of the email.  If you 
// provide neither of these, your email will have no body; if you provide both, SmartClient will
// ignore the file and just use the content directly provided.
// <p>
// <b>Mail server configuration</b><br>
// The mail server to use for sending emails is configured in the +link{server_properties,server.properties}
// file.  The following values can be provided:<p>
// <code>mail.system.mail.smtp.host</code>: The name of the SMTP server to use; defaults 
// to "localhost"<br>
// <code>mail.system.mail.smtp.port</code>: What port is the MTA listening on; defaults to 25<br>
// <code>mail.system.mail.smtp.auth</code>: Whether this server requires authentication; 
// defaults to false<br>
// <code>mail.system.mail.smtp.user</code>: SMTP user, if authentication is on; no default<br>
// <code>mail.system.mail.smtp.password</code>: Password, if authentication is on; no default<br>
// <p>
// Note that we will also pass any other properties that start <code>mail.system</code> through
// to the underlying Javamail <code>Session</code> object, so you can use Javamail features
// that aren't explicitly exposed by SmartClient's mail support.  For example, many cloud-based
// SMTP providers require that you issue a STARTTLS command before authenticating; you can 
// achieve this by adding the following line to +link{server_properties,server.properties}:<p>
// <code>mail.system.mail.smtp.starttls.enable: true</code>
//
// @treeLocation Client Reference/Data Binding/DataSource
// @visibility external
// @serverDS only
// @group mail
//<

//> @attr mail.templateFile (URL : null : IR)
// Qualified name of the file containing the message template, relative to webroot.  Like all
// the other <code>String</code> properties of <code>Mail</code>, you can use Velocity
// substitution variables in this property.  The content itself (ie, the contents of the file
// named in this property) can also contain Velocity substitutions, of course.
// <p>
// You must specify either this property or +link{mail.messageTemplate,messageTemplate}, but
// clearly it makes no sense to specify them both.
//
// @group mail
// @serverDS only
// @visibility external
//<

//> @attr mail.messageTemplate (String : null : IR)
// The text that will form the body of the mail message.  Like all other <code>String</code> 
// properties of <code>Mail</code>, you can use Velocity substitution variables in this property.
// <p>
// You must specify either this property or +link{mail.templateFile,templateFile}, but clearly
// it makes no sense to specify them both.
//
// @group mail
// @serverDS only
// @visibility external
//<

//> @attr mail.to (String : null : IR)
// Comma-separated list of recipients.  Like all other <code>String</code> 
// properties of <code>Mail</code>, you can use Velocity substitution variables in this property.
// <p>
// If you specify a "To:" line in the body of your email, recipient addresses will be parsed 
// and this property will be ignored.
//
// @group mail
// @serverDS only
// @visibility external
//<

//> @attr mail.cc (String : null : IR)
// Comma-separated list of "CC" recipients.  Like all other <code>String</code> 
// properties of <code>Mail</code>, you can use Velocity substitution variables in this property.
// <p>
// If you specify a "Cc:" line in the body of your email, recipient addresses will be parsed 
// and this property will be ignored.
//
// @group mail
// @serverDS only
// @visibility external
//<

//> @attr mail.bcc (String : null : IR)
// Comma-separated list of "BCC" recipients.  Like all other <code>String</code> 
// properties of <code>Mail</code>, you can use Velocity substitution variables in this property.
// <p>
// If you specify a "Bcc:" line in the body of your email, recipient addresses will be parsed 
// and this property will be ignored.
//
// @group mail
// @serverDS only
// @visibility external
//<

//> @attr mail.from (String : null : IR)
// The email address the message will appear to come from.  Like all other <code>String</code> 
// properties of <code>Mail</code>, you can use Velocity substitution variables in this property.
// <p>
// If you specify a "From:" line in the body of your email, the from address will be parsed 
// and this property will be ignored.
// <p>
// Ordinarily, this property (or the parsed equivalent in the email body) is mandatory for all
// emails.  However, if you specify the property <code>mail.system.mail.smtp.from</code> in 
// <code>server.properties</code>, it will become the default "from" address, and this property
// becomes optional.
//
// @group mail
// @serverDS only
// @visibility external
//<

//> @attr mail.replyTo (String : null : IR)
// An email address to indicate as the reply-to address on the message.  Like all other 
// <code>String</code> properties of <code>Mail</code>, you can use Velocity substitution 
// variables in this property.
//
// @group mail
// @serverDS only
// @visibility external
//<

//> @attr mail.subject (String : null : IR)
// The string to appear in the subject line of the message.  Like all other <code>String</code> 
// properties of <code>Mail</code>, you can use Velocity substitution variables in this property.
// <p>
// If you specify a "Subject:" line in the body of your email, the subject will be parsed 
// and this property will be ignored.
//
// @group mail
// @serverDS only
// @visibility external
//<

//> @attr mail.multiple (boolean : null : IR)
// By default, multiple mail messages are sent if the dsResponse contains multiple records. 
// Set this property to false to prevent this behavior.
//
// @group mail
// @serverDS only
// @visibility external
//<

//> @attr mail.messageData (VelocityExpression : null : IRA)
// Allows you to specify an alternative source of substitution values for the templating 
// process (by default, the data record(s) returned by the DataSource operation are used).
// If set, this +link{type:VelocityExpression} should evaluate to a Java <code>Map</code>
// object, or a Java <code>List</code> containing only <code>Map</code>s.
// <p>
// For example, assuming you had suitable context stored in a <code>Map</code> that is
// held in the current session as attribute "foo", the following would cause the mail 
// templating system to look in there for values when running substitution:
// <pre>
//     messageData="$session.foo"
// </pre>
//
// @group mail
// @serverDS only
// @visibility external
//<

//> @attr mail.contentType (String : null : IR)
// Used to override the content-type header applied to the email.  Primarily used to send 
// HTML emails rather than plain text ones (use "text/html" to do this).
//
// @group mail
// @serverDS only
// @visibility external
//<

//> @attr mail.encoding (String : null : IR)
// Used to override the character encoding we apply to the email body.  If you do not set 
// this property, it is derived from the +link{server_properties,server.properties} setting 
// <code>mail.system.default.encoding</code>, which is set to "UTF-8" by default.  This
// setting should be appropriate for most use cases.
//
// @group mail
// @serverDS only
// @visibility external
//<

    

        
	// Schema handling
	// --------------------------------------------------------------------------------------------

    // look up a schema, prefering the local schemaNamespace if we're namespaced
    getSchema : function (schemaName, schemaType) {
        // if we have a SchemaSet, look up through that
        var schemaSet = this.getSchemaSet();
        if (schemaSet != null) {
            //this.logWarn("looking up schema: " + schemaName + " of kind: " + schemaType + 
            //             " in schema namespace: " + this.schemaNamespace);

            var schema = schemaSet.getSchema(schemaName, schemaType);
            if (schema != null) return schema;
        }

        // look up schema through our WSDL web service, if we have one, to allow for multiple
        // SchemaSets within a WebService that reference between each other (SForce service is
        // like this, for the SObject type)
        var service = this.getWebService();
        if (isc.isA.WebService(service)) return service.getSchema(schemaName, schemaType);

        // finally, look for any globally defined schema
        return isc.DS.get(schemaName, null, null, schemaType);
    },

    // how to refer to record[s] of this DataSource
    getTitle : function () {
        return this.title || this.ID;
    },
    getPluralTitle : function () {
        return this.pluralTitle || (this.getTitle() + "s");
    },

    // get the field best used as a title for records from this DataSource 
    defaultTitleFieldNames:[
    	"title",
    	"label",
    	"name",
    	"id"
    ],
    getTitleField : function () {
        if (this.titleField == null) {
            // if a title field hasn't been explicitly specified, take a guess.
            // Also, remember the guess (this is an inner loop)
            var fieldNames = isc.getKeys(this.getFields());
            var normalizedFieldNames = fieldNames.map("toLowerCase");
            for (var i = 0; i < this.defaultTitleFieldNames.length; i++) {
            	var index = normalizedFieldNames.indexOf(this.defaultTitleFieldNames[i]);
            	if (index != -1 && !this.getField(fieldNames[index]).hidden) {
            		this.titleField = fieldNames[index];
            		break;
				}
			}
			// Use the first (non hidden) fieldName
			if (this.titleField == null) {
				for (var i = 0; i < fieldNames.length; i++) {
					if (this.getField(fieldNames[i]).hidden) continue;
					this.titleField = fieldNames[i];
					break;
				}
				if (this.titleField == null) this.titleField = fieldNames[0];
        	}
        }
        return this.titleField; 
    },

    getIconField : function () {
        var undef;
        if (this.iconField === undef) {
            this.iconField = null;

            // if an Icon field hasn't been explicitly specified, take a guess.
            // Also, remember the guess (this is an inner loop)
            var fieldNames = isc.getKeys(this.getFields());
            var iconFieldNames = ["picture", "thumbnail", "icon", "image", "img"];
            for (var i = 0; i < iconFieldNames.length; i++) {
                var fieldName = iconFieldNames[i],
                    field = this.getField(fieldName);
                if (field && isc.SimpleType.inheritsFrom(field.type,"image")) {
                    this.iconField = fieldName;
                }
            }
        }
        return this.iconField;
    }, 
    
	// init the various sources for a view
	// 	this loads those datasources and creates a "fields" array that is
	//	the union of the fields coming from the other sources
	initViewSources : function () {
		var list = this.fields = {};
		
		for (var sourceName in this.sources) {
			var ds = isc.DS.get(sourceName);
			if (!ds) continue;
			
			var sourceFields = this.sources[sourceName].fields;
			for (var field in sourceFields) {
				var value = sourceFields[field],
					item = null;

				if (value == "*") {
					// if value is a "*", field is the name of the field in the datasource
					item = ds.fields[field];
				} else if (isc.isA.String(value)) {
					// if value is some other string, value is the name of some other field in ds
					item = ds.fields[value];
				} else if (isc.isAn.Object(value)) {
					// it's an object, first apply properties in ds[value.field], then apply
					//	the properties in the value to override
					item = isc.addProperties({}, ds.fields[ds.fields[value.field]]);
					isc.addProperties(item, value);
				}
				
				if (item) list[field] = item;
			}
		}
	},

    inheritsSchema : function (ds) {
        if (ds == null) return false;
        if (isc.isA.String(ds)) ds = this.getSchema(ds);

        if (ds == this || ds == isc.DS.get("Object")) return true;

        if (!this.hasSuperDS()) return false;
        return this.superDS().inheritsSchema(ds);
    },

    getInheritedProperty : function (property) {
        if (this[property]) return this[property];
        var superDS = this.superDS();
        return superDS ? superDS.getInheritedProperty(property) : null;
    },

	hasSuperDS : function () {
		if (this.inheritsFrom) return true;
		return false;
	},

	superDS : function () {
		if (this.hasSuperDS()) return this.getSchema(this.inheritsFrom);
		return null;
	},
		
    //> @method dataSource.getField()
    // Return the field definition object.
    // @param fieldName (String) Name of the field to retrieve
    // @return (DataSourceField) field object
    // @visibility external
    //<
    getField : function (fieldName, checkDataPath) {
        if (isc.isAn.Object(fieldName)) fieldName = fieldName.name;
        var fields = this.getFields();
        var field = fields ? fields[fieldName] : null;
		
        if (field == null && checkDataPath && fields != null) {
        	for (var i in fields) {
        		if (fields[i].dataPath == fieldName) {
        			field = fields[i];
        			break;
				}
        	}
        }
        return field;
    },
		
    //> @method dataSource.getFieldForDataPath()
    // Return the field definition object corresponding to the supplied dataPath
    // @param dataPath (String) dataPath of the field to retrieve
    // @return (DataSourceField) field object, or null if no field corresponds to the 
    //                           supplied dataPath
    // @visibility external
    //<
    getFieldForDataPath : function (dataPath) {
        if (isc.isAn.Object(dataPath)) dataPath = dataPath.dataPath;
        if (!dataPath) return null;
        var segments = dataPath.trim(isc.Canvas._$slash).split(isc.Canvas._$slash);
        var dataSource = this;
        for (var i = 0; i < segments.length; i++) {
        	if (dataSource == null) return null;
            var field = dataSource.getField(segments[i]);
            if (!field) return null;
            dataSource = isc.DataSource.get(field.type);
        }
        return field;
    },
    
    // Given a dataPath referring to fields within nested dataSources, return the
    // dataSource associated with the dataPath records.
    // pathToRecord: Does the dataPath include the field within a record, or is it 
    // the path to a record as a whole (for example the dataPath applied to an entire dataBoundComponent?)
    getDataSourceForDataPath : function (dataPath, pathToRecord) {
        if (isc.isAn.Object(dataPath)) dataPath = dataPath.dataPath;
        if (!dataPath) return null;
        var segments = dataPath.trim(isc.Canvas._$slash).split(isc.Canvas._$slash);
        var dataSource = this;
        // Note that we don't need to iterate all the way to the end of the dataPath, just to the
        // penultimate entry - we don't care about the type of the field itself, just the DS it's
        // nested inside.
        var length = pathToRecord ? segments.length : segments.length-1;
        for (var i = 0; i < length; i++) {
            var field = dataSource.getField(segments[i]);
            if (!field) {
            	this.logInfo("getDataSourceForDataPath() - unable to find nested field:"
            			+ segments[i] + " from dataSource:" + dataSource +
            			". Orginal dataPath:" + dataPath, "dataBinding");
            	return null;
			}
			var parentDS = dataSource;
            dataSource = isc.DataSource.get(field.type);
            if (dataSource == null) {
                this.logInfo("getDataSourceForDataPath() - unable to find nested dataSource for field:" + 
                	field.name + " of type:" + field.type + " on dataSource:" + parentDS +
                	". Original dataPath:" + dataPath, "dataBinding");
                break;
            }
        }
        return dataSource;
    	
    },

    getFieldByTitle : function (fieldTitle) {
        var fields = isc.getValues(this.getFields());
        
        for (var i=0; i<fields.length; i++) {
            var field = fields[i],
                title = field.title || isc.DS.getAutoTitle(fields[i].name);
            if (title == fieldTitle) return field;
        }

        return null;
    },
    
    //> @method dataSource.getDisplayValue()
    // Given a fieldName and a dataValue, apply any +link{DataSourceField.valueMap} for the
    // field and return the display value for the field
    // @param fieldName (String) name of the field to retrieve a value for
    // @param value (any) data value for the field
    // @return (any) display value for the field
    // @visibility external
    //<
    getDisplayValue : function (fieldName, value) {
        var field = this.getField(fieldName);
        if (field == null) return value;
        if (isc.isAn.Object(field.valueMap) && !isc.isAn.Array(field.valueMap) &&
            isc.propertyDefined(field.valueMap, value))
        {
            return field.valueMap[value];
        }
        return value;
    },

    //> @method dataSource.getFieldNames()
    // Retrieves the list of fields declared on this DataSource.
    //
    // @param excludeHidden (boolean) If true, returns only those fields that are 
    //                                not marked as hidden
    // @return (Array of String) names of all fields declared on this DataSource
    // @visibility external
    //<
    getFieldNames : function (excludeHidden) {
        if (isc._traceMarkers) arguments.__this = this;
        if (!excludeHidden) return isc.getKeys(this.getFields());
        var fields = this.getFields(),
            filtered = [],
            filteredIndex = 0;
        for (var fieldName in fields) {
            if (fields[fieldName] != null && !fields[fieldName].hidden) {
            	filtered[filteredIndex++] = fieldName;
            }
        }
        return filtered;
    },

    // get the field defined on this DataSource only (as opposed to inherited)
	getLocalFields : function (dontResolveTypes) {
        if (this._builtinValidatorsAdded) return this.fields;
        if (dontResolveTypes) return this.fields;
        this._addTypeDefaults();
        this._autoDeriveTitles();
        this._builtinValidatorsAdded = true;
        return this.fields;
	},
    
	getFields : function () {
        if (isc._traceMarkers) arguments.__this = this;
    
		if (this.mergedFields) return this.mergedFields;

        //this.logWarn("field init for: " + this);

        // NOTE: silently ignore looping schema definitions.  Currently can happen with
        // schemaTranslator if an element is declared to be of a complexType named the same as
        // the element.
        if (!this.hasSuperDS() || this == this.superDS()) {
            return this.mergedFields = this.getLocalFields();
        }

        // Log a warning and bail if the superDS does not exist, in order to avoid a hard 
        // crash a few lines further down when we dereference it
        // the element.
        if (!this.superDS()) {
            this.logWarn("DataSource " + this.ID + " inheritsFrom " + this.inheritsFrom + 
                         ", but there is no DataSource of that name currently loaded. " + 
                         "Ignoring the inheritsFrom declaration.");
            return this.mergedFields = this.getLocalFields();
        }

        // combine fields: local fields override superDS fields.  
        

        // field inheritance flags:
        // - [default] local fields first, then parent fields
        // - useParentFieldOrder: fields are in parent order
        // - showLocalFieldsOnly: only local fields are shown.  Parent fields are present but
        //   hidden so that serialization is correct.  Typical setting when extending imported
        //   schema to create a DataSource that serves as a default field set for DataBound
        //   components
        // - restrictToLocalFields: use internally for xs:restriction, in which parent fields
        //   must really be dropped (not just hidden) if they aren't redeclared - they aren't
        //   allowed when serializing
        var superDS = this.superDS();
        	
        /*
        // future plans: use combineFieldOrders, equivalent to 
        // dataBoundComponent.useAllDataSourceFields:true, as default binding.
        if (!this.showLocalFieldsOnly && !this.restrictToLocalFields &&
            !this.useParentFieldOrder) 
        {
            return this.mergedFields = 
                this.combineFieldOrders(superDS.getFields(), 
                                        isc.getValues(this.getLocalFields()));
        }
        */

        if (this.showLocalFieldsOnly || this.restrictToLocalFields) {
            this.useParentFieldOrder = false;
        }

        var localFields = isc.addProperties({}, this.getLocalFields()),
            fields;
        if (!this.useParentFieldOrder) {
            // put together fields by adding super fields to the local fields, so local fields
            // are first
            fields = localFields;
        } else {
            // put together fields by adding fields in superDS field order, and adding the rest
            // of the local fields at the end
            fields = {};
        }
    
        // if we have a superDS, add it's fields here.  NOTE: we do this in this order so
        // that locally defined fields appear first in the merged list, which affects the
        // default display of editors
        var fieldNames = (this.restrictToLocalFields ? isc.getKeys(this.getLocalFields()) : 
                                                       superDS.getFieldNames());
        for (var i = 0; i < fieldNames.length; i++) {
            var fieldName = fieldNames[i],
                localField = localFields[fieldName];

            if (localField != null) {
                var superField = superDS.getField(fieldName);
                //this.logWarn("local and super definition for field: " + fieldName +
                //             ", local field: " + this.echo(localField) +
                //             ", super field: " + this.echo(superField));
                // if a field is present in a sub-DS which is hidden in a superDS,
                // assume the intent was to expose it, unless "hidden='true'" is reiterated, or
                // some other property that would hide the field is specified.  Likewise for
                // visibility=internal.  Note, this doesn't apply in cases where where we are
                // inheriting because of the presence of autoDeriveSchema and/or schemaBean.
                // In that case, we want to inherit everything that is not expressly overridden
                // on this DS, since the only purpose of the parent DS in this case is to 
                // provide default settings for this specific DS.
                
                if (superField.hidden && localField.hidden == null && 
                    !localField.inapplicable && !this.autoDeriveSchema && !this.schemaBean) 
                {
                    //this.logWarn("dataSource: " + this + 
                    //             " overrides hidden field: " + fieldName +
                    //             " in superDS: " + superDS);
                    localField.hidden = false;
                }
                if (superField.visibility != null && localField.visibility == null &&
                    !localField.inapplicable && !localField.hidden && 
                    superField.visibility == "internal")
                {
                    //this.logWarn("dataSource: " + this + 
                    //             " overrides internal field: " + fieldName +
                    //             " in superDS: " + superDS);
                    localField.visibility = "external";
                }
                
                // Save a copy of the localField - it's about to be clobbered
                var localCopy = isc.addProperties({}, localField);
                
                // local and super field definition: combine with overrides
                fields[fieldName] = superDS.combineFieldData(localField);
                
                // Special override case: all fields have a title attribute, because one will
                // auto-derived from the name if necessary.  However, we do not want to use 
                // the auto-derived title if a better one can be inherited
                if (localCopy._titleAutoDerived) fields[fieldName].title = superField.title;
            } else {
                // field definition in parent only
                if (this.showLocalFieldsOnly) {
                    fields[fieldName] = isc.addProperties({}, superDS.getField(fieldName));
                    fields[fieldName].hidden = "true";
                } else {
                    fields[fieldName] = superDS.getField(fieldName);
                }
            }
            // check off this local field as having appeared in the parent already
            if (this.useParentFieldOrder) delete localFields[fieldName];
		}
        // add fields not declared in the parent
        if (this.useParentFieldOrder) isc.addProperties(fields, localFields);
    
        // special rule for XML schema: an xs:restriction is like showLocalFieldsOnly *except*
        // that attributes need not be redeclared to be valid in the derived type.
        // NOTE field order doesn't matter here; attributes are always at the end.
        if (this.restrictToLocalFields && isc.Schema && isc.isA.Schema(this)) {
            var parentFieldNames = superDS.getFieldNames();
            for (var i = 0; i < parentFieldNames.length; i++) {
                var fieldName = parentFieldNames[i],
                    parentField = superDS.getField(fieldName);

                if (parentField.xmlAttribute) {
                    fields[fieldName] = fields[fieldName] || parentField;
                }
            }
        }

		return this.mergedFields = fields;
	},
	
	// helper method to tell us if a dataSource has fields. Helpful if a ds
	// doesn't define fields itself, but inherits them instead. Used in 
	// listGrid.setFields
	hasFields : function () {
	    if (this.fields) return true;
	    else if (this.inheritsFrom) {
	        // climb the inheritance chain and check for fields
	        var ds = this;
	        while (ds.inheritsFrom) {
	            ds = isc.DataSource.get(this.inheritsFrom);
	            if (ds.fields) return true;
	        }
	    }
	    return false;
	},
	
    // get all simple type fields, throughout this DataSource and any sub-DataSources.
    // if path/pathProperty is passed, also construct a dataPath and return *copies* of the
    // discovered fields with the dataPath applied under "pathProperty".
    getFlattenedFields : function (flatFields, path, pathProperty) {
        flatFields = flatFields || {}; // if called recursively, just add new fields to the
                                       // existing listing
        var fieldNames = this.getFieldNames();

        for (var i = 0; i < fieldNames.length; i++) {
            var fieldName = fieldNames[i],
                field = this.getField(fieldName);
            if (!this.fieldIsComplexType(fieldName)) {
                // first field with a given name wins (not clear what's best here)
                if (flatFields[fieldName] == null) {
                    // label the field with the DataSource it came from
                    field.sourceDS = this.ID; 
                    if (path) {
                        field = isc.addProperties({
                        }, field);
                        field[pathProperty] = path + "/" + fieldName;
                    }
                    flatFields[fieldName] = field;
                }
            } else {
                var nestedDS = this.getFieldDataSource(field);
                if (path != null) path = (path ? path + "/" : "") + fieldName;
                nestedDS.getFlattenedFields(flatFields, path, pathProperty);
            }
        }
        return flatFields;
    },

    fieldIsComplexType : function (fieldName) {
        var field = this.getField(fieldName);
        if (field == null) return false;
        return (field.type != null && !field.xmlAttribute && this.getSchema(field.type) != null) 
            || this.fieldIsAnonDataSource(field);
    },

    fieldIsAnonDataSource : function (field) {
        if (!field.fields) return false;
        var fields = isc.isAn.Array(field.fields) ? field.fields : isc.getValues(field.fields);
        return fields.length > 0 && isc.isAn.Object(fields.get(0));
    },

    getFieldDataSource : function (field, schemaType) {
        // bail if no field passed
        if (!field) return null;

        if (this.fieldIsAnonDataSource(field)) {
            // anonymous datasource field
            if (!field._anonDataSource) {
                // ds hasn't been gen'd before - do so now and cache on the field
                var ds = isc.DataSource.create({ "class": "DataSource", fields: field.fields });
                field._anonDataSource = ds;
            }
            // return the cached ds
            return field._anonDataSource;
        } 

        return field.type != null ? this.getSchema(field.type, schemaType) : null;
    },

    // find an XML tag that has the requested type, anywhere in the XML structure that this
    // DataSource and it's child tags represents.  Used for putting together a default
    // recordXPath when calling web services and probably not appropriate for anything else,
    // as the returned tagName isn't a path, just a tagName
    
    findTagOfType : function (targetType, parentSchema, parentFieldName) {
        //this.logWarn("findTagOfType: searching for targetType: " + targetType);
        // search for immediate fields of the target type
        var fieldNames = this.getFieldNames();
        for (var i = 0; i < fieldNames.length; i++) {
            var fieldName = fieldNames[i],
                field = this.getField(fieldName);

            //this.logWarn("findTagOfType: field: " + fieldName + " of type: " + field.type +
            //             (field.type == targetType ? " is a match" : 
            //                this.fieldIsComplexType(fieldName) ? 
            //                    "(complextype)" : "(simpleType)"));

            if (field.type == targetType) return [this, fieldName, parentSchema, parentFieldName];
    
            // for complex type fields, check if they in turn have fields of the target type
            if (this.fieldIsComplexType(fieldName)) {
                //this.logWarn("recursing into childTag: " + this.echo(childTag));
                var childDS = this.getFieldDataSource(field),
                    childResult = childDS.findTagOfType(targetType, this, fieldName);
                if (childResult) return childResult;
            }
        }
    }, 

    // XML Text Content
    // ---------------------------------------------------------------------------------------
    

    // property to use to store textContent derived from an XML element that also has
    // attributes.  This currently only works for an XML element that has no element children.
    textContentProperty:"xmlTextContent",

    // return the field definition to be used for the text content of an XML element.
    // You can create a field for the textContent (in order to provide type) by just naming a
    // field after the textContentProperty.
    // The schemaTranslator, when encountering <xs:simpleContent>, creates a special field 
    // xsdSimpleContent and sets textContentProperty to match.
    getTextContentField : function () {
        return this.getField(this.textContentProperty);
    },

    // in the absence of an explicit textContentField
    hasXMLElementFields : function (textContentProperty) {
        textContentProperty = textContentProperty || this.textContentProperty;
        var fieldNames = this.getFieldNames();
        for (var i = 0; i < fieldNames.length; i++) {
            if (fieldNames[i] == textContentProperty) continue;
            if (this.getField(fieldNames[i]).xmlAttribute) continue;
            return true;
        }
        return false;
    },

    // EditMode
    // ---------------------------------------------------------------------------------------

    //>EditMode get the list of groups defined in the DataSource, which is the order the Groups
    // should appear in the component editor
    getGroups : function () {
        var ds = this;
        while (ds.groups == null && ds.hasSuperDS()) ds = ds.superDS();
        return ds.groups;
    },
    //<EditMode

    // return a field of the type specified by the dataSource ID passed in, searching in order of
    // subclass to superclass
    // includeAllFields: boolean; if true, advanced fields will be included in the search (default
    //   behavior is to omit them)
    // excludedFields: a map of fields to be excluded from the search (keys and values are the
    //   field names)
    _$Defaults : "Defaults",
    _$Properties : "Properties",
    getObjectField : function (targetType, includeAllFields, excludedFields) {
    	
        if (!targetType) return null;

        // search through fields in reverse order to find the best match
        var fields = this.getLocalFields(),
            fieldNames = isc.getKeys(fields).reverse(),
            classForType = isc.DataSource.getNearestSchemaClass(targetType)
        ;
        
        
        if (excludedFields == null) excludedFields = {};
        
        // search for the field that has the smallest inheritance distance to the test class
        var nearestDistance = -1,
            nearestMatch = null
        ;

        for (var i = 0; i < fieldNames.length; i++) {
            var fieldName = fieldNames[i],
                field = fields[fieldName],
                distance
            ;

            // HACK: ignore fields intended for properties to configure subobjects
            if (isc.endsWith(fieldName, this._$Properties) || 
                isc.endsWith(fieldName, this._$Defaults)) continue;

            // remember excluded fields, so we can pass the list to our superclass, so that a
            // we won't choose a field in a superclass that was marked "inapplicable" in a
            // subclass
            if (!includeAllFields && 
                (excludedFields[fieldName] || 
                 field.advanced || field.inapplicable || field.hidden ||
                 (field.visibility != null && field.visibility == "internal"))) 
            {
                excludedFields[fieldName] = fieldName;
                continue;
            }

            // If there is no class for the targetType, 
            // then don't try to find a best match based on inheritance distance, just find any
            // matching field
            if (!classForType && field.type == targetType) return fieldName;

            // find the field that is most specific to the targetType by evaluating inheritance
            // distance between the field type and the targetType
            if (classForType && classForType.isA(field.type)) {
                distance = isc.DS.getInheritanceDistance(field.type, targetType);
                //this.logWarn("field " + fieldName + 
                //             " has distance from type: " + distance); 
                if (nearestMatch == null || distance < nearestDistance) {
                    nearestMatch = fieldName;
                    nearestDistance = distance;
                }
            }
        }
        
        // match was found in the local fields of this datasource; if it has a superDS, get
        // the best match for the superDS and compare the inheritance distances, returning
        // the match with the smaller distance (if the distances are equal, return the match
        // from the local fields)
        if (nearestMatch != null) {
            if (nearestDistance == 0 || !this.hasSuperDS()) {
                return nearestMatch;
            } else {
                var superDSMatch = this.superDS().getObjectField(
                                        targetType, includeAllFields, excludedFields);
                if (superDSMatch) {
                    var fieldType = this.getField(superDSMatch).type,
                        superDSMatchDistance = isc.DS.getInheritanceDistance(fieldType,
                                                                             targetType);
                }
                
                return (superDSMatch && (superDSMatchDistance < nearestDistance)) 
                        ? superDSMatch : nearestMatch;
            }
        // no match was found in the local fields of this datasource, so return the best match
        // found for its superDS, if it has one (recursive search)
        } else if (this.hasSuperDS()) {
            return this.superDS().getObjectField(targetType, includeAllFields, excludedFields);
        }

        return null;
    },
    
        
    // Relationship detection 
	// --------------------------------------------------------------------------------------------
    // primary keys, foreign keys, within-DS relations like trees, cross-DS relations

	getLocalPrimaryKeyFields : function () {
		if (!this.primaryKeys) {
			this.primaryKeys = {};
			var fields = this.getFields();
			for (var fieldName in fields) {
				var fieldSpec = fields[fieldName];
				if (fieldSpec.primaryKey) {
					this.primaryKeys[fieldName] = fieldSpec;
				}
			}
		}
		return this.primaryKeys;
	},

    filterPrimaryKeyFields : function (records) {
        var fieldMask = this.getPrimaryKeyFields();
        return isc.applyMask(records, isc.getKeys(fieldMask));
    },

    filterDSFields : function (records) {
        var fieldMask = this.getFields();
        return isc.applyMask(records, isc.getKeys(fieldMask));
    },

    recordHasAllKeys : function (record) {
        var fields = this.getPrimaryKeyFields();
        for (var fieldName in fields) {
            if (record[fieldName] == null) return false;
        }
        return true;
    },

    // Given a foreign record and a foreignDS, returns a map where the keys are the names of
    // the fields in this datasources that are marked as foreignKey into the foreignDS and the
    // values are the foreignRecord values for the fields in the foreignDS that our foreignKeys
    // point into.
    getForeignKeysByRelation : function (foreignRecord, foreignDS) {
        // Figure out the foreignKeys of the ds that will store the binary value
        var foreignKeyFields = this.getForeignKeyFields(foreignDS);
        if (!foreignKeyFields) return {};

        // grab the foreignRecord values for our foreignKeys
        var record = {};
        for (var foreignKeyFieldName in foreignKeyFields) {
            var foreignKeyField = foreignKeyFields[foreignKeyFieldName];
            var foreignFieldName = isc.DataSource.getForeignFieldName(foreignKeyField);

            // don't return an undef or null for a primary key, just omit it
            var value = foreignRecord[foreignFieldName];
            if (value || value === 0) record[foreignKeyFieldName] = value;
        }
        return record;
    },
    
    
    //>@method DataSource.getPrimaryKeyFields()
    // Returns this DataSource's +link{DataSourceField.primaryKey,primaryKey} fields as a map
    // of fieldName to field.
    // @return (Record) Javascript object containing all this datasource's primaryKey fields,
    //                  as a map of field name to field
    // @see DataSource.getPrimaryKeyField
    // @see DataSource.getPrimaryKeyFieldNames
    // @visibility external
    //<
	getPrimaryKeyFields : function () {
		if (!this.mergedPrimaryKeys) {
			this.mergedPrimaryKeys = {};

			// if we have a superDS, merge those primaryKeys in first
			if (this.hasSuperDS()) {
				isc.addProperties(this.mergedPrimaryKeys, this.superDS().getPrimaryKeyFields());
			}

			// add local fields
			isc.addProperties(this.mergedPrimaryKeys, this.getLocalPrimaryKeyFields());			
		}
		return this.mergedPrimaryKeys;
	},

    // return all foreignKey fields defined on this ds.  If a foreignDS is specified, return
    // only the foreignKeys that point into the foreignDS
    getForeignKeyFields : function (foreignDS) {
        if (isc.isA.DataSource(foreignDS)) foreignDS = foreignDS.ID;

        var fields = this.getFields();
        if (!fields) return null;

        var foreignKeyFields = {};
        for (var fieldName in fields) {
            var field = fields[fieldName];
            if (field.foreignKey) {
                if (foreignDS) {
                    var fieldForeignDS = isc.DataSource.getForeignDSName(field, (foreignDS || this));
                    if (fieldForeignDS != foreignDS) continue;   
                }
                foreignKeyFields[field.name] = field;
            }
        }
        return foreignKeyFields;
    },

	getLocalPrimaryKeyFieldNames : function () {
		var localPrimaryKeyFields = this.getLocalPrimaryKeyFields();
		var localFieldNames = [];
		for (var fieldName in localPrimaryKeyFields) {
			localFieldNames.add(fieldName);
		}
		return localFieldNames;
	},

    //>@method DataSource.getPrimaryKeyFieldNames()
    // Returns a list of the names of this DataSource's 
    // +link{DataSourceField.primaryKey,primaryKey} fields.
    // @return (Array of String) The list of the names of this datasource's primaryKey fields
    // @see DataSource.getPrimaryKeyFields
    // @visibility external
    //<
	getPrimaryKeyFieldNames : function () {
        return isc.getKeys(this.getPrimaryKeyFields());
    },
    
    //>@method DataSource.getPrimaryKeyField()
    // Returns a pointer to the primaryKey field for this DataSource.  If this dataSource has
    // a composite primary key (ie, multiple primaryKey fields), returns just the first 
    // primaryKey field.
    // @return (DataSourceField) primary key field object
    // @see DataSource.getPrimaryKeyFields
    // @visibility external
    //<
    getPrimaryKeyField : function () {
        var pks = this.getPrimaryKeyFields();
        for (var field in pks) {
            return pks[field];
        }
    },
    
    //>@method DataSource.getPrimaryKeyFieldName()
    // Returns the primary key fieldName for this DataSource.    If this dataSource has
    // a composite primary key (ie, multiple primaryKey fields), returns just the first 
    // primaryKey field name.
    // @return (string) primary key field name
    // @see DataSource.getPrimaryKeyFieldNames
    // @visibility external
    //<
    getPrimaryKeyFieldName : function () {
        return this.getPrimaryKeyFieldNames()[0];
    },


    // Relationships to other DataSources
	// --------------------------------------------------------------------------------------------
    addChildDataSource : function (dataSource) {
        var children = this._childDataSources = (this._childDataSources || []);
        children.add(dataSource);
    },

    getChildDataSources : function () {
        return this._childDataSources;
    },
    
    getChildDataSource : function (excludeSelf) {
        var childDSs = this.getChildDataSources();
        if (childDSs == null) return null;
        var dataSource;
        for (var i = 0; i < childDSs.length; i++) {
            if (!childDSs[i] || (excludeSelf && childDSs[i] == this)) continue;
            if (!dataSource) {
                dataSource = childDSs[i];
                // don't break - we want to notify the user when this method is returning
                // an ambiguous result
            } else if (dataSource != childDSs[i]) {
                this.logInfo("getChildDatasource(): This DataSource has multiple child " +
                    "DataSources defined making getChildDataSource() ambiguous. Returning the " +
                    "first child dataSource only - call getChildDataSources() to retrieve a " +
                     "complete list.");
                break;
            }
        }
        return dataSource
    },
    
    // get the default tree relationship declared for this DataSource, or null if no tree
    // relationship can be detected.
    // The returned relationship is of the form 
    //            { parentIdField : [fieldName], 
    //              idField : [fieldName],
    //              childrenProperty: [fieldName],
    //              parentDS : [dataSource name]}
    // XXX multi-field relations are not currently supported
    //
    // If parentDS is not passed in it will be autodected from either the first foreignKey found on
    // this DS or the one specified via foreignKeyFieldName (second arg).
    //
    // If foreignKeyFieldName is not passed in, we find the first field on this DS that has a
    // foreignKey definition and use that.
    // Note that a valid hierarchical dataSource should have either a foreignKey or 
    // childrenProperty field (or both) specified as otherwise we won't be able to link 
    // records as parents/children.
    getTreeRelationship : function (parentDS, foreignKeyFieldName) {
        
        // make sure we have DS instance
        if (isc.isA.String(parentDS)) parentDS = this.getSchema(parentDS);

        

        // if the name of the foreignKey wasn't passed in, autodetect it by looking for the first
        // field on this ds with a foreignKey pointing at the appropriate dataSource.
        var fields = this.getFields();
        if (foreignKeyFieldName == null) {
            for (var fieldName in fields) {
                var currentField = fields[fieldName];
                if (currentField.foreignKey != null) {
                    // If we were passed no parentDS and no foreignKeyFieldName, always use the
                    // first field with a specified foreignKey
                    if (!parentDS ||
                        (parentDS.getID() == isc.DataSource.getForeignDSName(currentField, this)))
                    {
                        foreignKeyFieldName = fieldName;
                        break;
                    }
                }
            }
        }
        
        var targetField;
        // if there was no foreignKey property specified on any of the fields, find the first
        // exact field name match between the two datasources.
        if ( foreignKeyFieldName == null && parentDS) {
            foreignKeyFieldName = targetField = isc.getKeys(this.fields).intersect(isc.getKeys(parentDS.fields))[0];
            this.logInfo("no foreign key declaration, guessing tree relationship " + 
                         "is on field name: " + foreignKeyFieldName + " which occurs in both DataSources");
        }

        var field;
        if (foreignKeyFieldName) field = fields[foreignKeyFieldName];

        //>DEBUG
        if (field == null) {
            // This is acceptable - it can occur children are served up as arrays directly on nodes.
            this.logDebug("getTreeRelationship(): Unable to find foreignKeyField." +
                          "foreignKeyFieldName specified as:" + foreignKeyFieldName);
        }
        //<DEBUG    

        // if parentDS was not passed in and we have a foreignKey field
        // parse it out of the foreignKey property of the field
        if (!parentDS) {
            if (!field) parentDS = this;
            else {
                var parentDSName = isc.DataSource.getForeignDSName(field, this);
                parentDS = this.getSchema(parentDSName);
            }
        }

        if (!targetField) targetField = field ? isc.DataSource.getForeignFieldName(field) : null;
            
        if (targetField == null) {
            // target field not specified; assume primary key.  Get the primary key of the
            // parent dataSource, limiting to one field
            var idField = parentDS.getPrimaryKeyFieldNames();
            if (isc.isAn.Array(idField)) {
                //>DEBUG
                if (idField.length > 1) {
                    this.logWarn("getTreeRelationship: dataSource '" + parentDS.ID + 
                                 "' has multi-field primary key, which is not " + 
                                 "supported for tree viewing.  Using field '" +
                                 idField[0] + "' as the only primary key field");
                }
                //<DEBUG
                idField = idField[0];
            }
            targetField = idField;
        }

        var isFolderField;
        var childrenProperty;

        // Suppoert dataSource.childrenField as well as the dataSourceField.childrenProperty 
        // boolean
        if (this.childrenField) childrenProperty = this.childrenField;

        for (fieldName in fields) {
            var field = fields[fieldName];
            // autodetect a field marked as the 'isFolderProperty' and 'childrenProperty' and
            // make that available via the relationship
            if (field.isFolderProperty) isFolderField = fieldName;
            if (field.childrenProperty) childrenProperty = fieldName;

            // By default, set field.multiple to true for the children field.
            // This expects XML type data to have explicit an array of child elements under
            // the childrenProperty node - like this:
            //   <item name=.../>
            //      <children>
            //          <item name=.../>
            //          <item name=.../>
            //      </children>
            //   </item>
            // However - if field.multiple is set to false, dont override it.
            // When this property is set to false, the server can still deliver an array of
            // elements by simply having a number of same-named child nodes - like this:
            //  <item name=.../>
            //      <child name=.../>
            //      <child name=.../>
            //  </item>
            if (childrenProperty == fieldName && (field.multiple == null)) {
                field.multiple = true;
            }
        }

        var relationship = {
            childDS : this,
            parentDS : parentDS,
            isFolderProperty: isFolderField
        }
        // the field on this DataSource with the foreignKey declaration
        // points to "idField" on the parent DataSource
        if (foreignKeyFieldName) {
            relationship.parentIdField = foreignKeyFieldName;
            relationship.idField = targetField;
        }
        if (childrenProperty) relationship.childrenProperty = childrenProperty;
        
        // If both foreignKey field name and childrenProperty are null there's no way we can
        // serve up children meaningfully:
        // This is not necessarily an error - this method may be called when there is no
        // foreignKey relationship -- EG databound treeGrid drop
        //>DEBUG
        if (childrenProperty == null && foreignKeyFieldName == null) {
            this.logInfo("getTreeRelationship(): No specified foreignKeyField or childrenProperty.")
        }
        //<DEBUG

        // determine the rootValue.  This applies only to within-DataSource trees; for
        // cross-DataSource trees, in effect all records of the parentDS are roots.
        // rootValue defaults to null.
        if (parentDS == this) {
            var rootValue = foreignKeyFieldName ? this.getField(foreignKeyFieldName).rootValue : null;
            
            if (rootValue == null) relationship.rootValue = null;
            else relationship.rootValue = rootValue;
        }

        // NOTE: there may be more than one tree relationship.  For now we're just
        // auto-detecting the first.
        
        return relationship;
    },

    // DataBinding support
	// --------------------------------------------------------------------------------------------
 
    _$name:"name",
    combineFieldOrders : function (parentFields, localFields, shouldUseFieldFunc) {
        // all the fields in the DataSource that would normally be shown will be shown in the
        // order they occur in the DataSource.  The specified fields act as overrides.
        // NOTE: in this mode, if you specify any non-DataSource fields, they appear right
        // after the last DataSource field you added.  In this way you can add extra
        // fields, or items like spacers, at specific positions with respect to DataSource
        // fields.

        // create a new array to hold the combined field specifications
        var combinedFields = [];

        // add any non-DS fields before the first DS field
        this._addNonDSFields(localFields, 0, parentFields, combinedFields, shouldUseFieldFunc);

        // go through all dataSource fields
		for (var fieldName in parentFields) {
            var parentField = parentFields[fieldName],
                // look for a localField with matching name (this is slow but not time
                // critical)
                localFieldIndex = 
                    localFields.findIndex(this._$name, fieldName);

            if (localFieldIndex != -1) {
                // combine the specified field and the reference dsField
                var localField = localFields[localFieldIndex],
                    combinedField = this.combineFieldData(localField);

                // check if this is a field this component should use.  NOTE: we check this
                // after combining, so that you can suppress a dsField by adding hidden:true
                if (shouldUseFieldFunc == null ||
                    shouldUseFieldFunc(combinedField, this, true)) combinedFields.add(combinedField);

                // if there are any non-dataSource fields specified after this dsField, add them
                // here
                this._addNonDSFields(localFields, localFieldIndex + 1,
                                     parentFields, combinedFields, shouldUseFieldFunc);
            } else {
                // no specified field data for this dsField, just clone the dsField
                if (shouldUseFieldFunc == null ||
                    shouldUseFieldFunc(parentField, this)) {
                    combinedFields.add(isc.addProperties({}, parentField));
                }
            }
		}
        return combinedFields;
    },

    // starting at position, add in any fields that don't refer to DataSource fields
    _addNonDSFields : function (localFields, position, parentFields, combinedFields,
                                shouldUseFieldFunc) 
    {
        for (var i = position; i < localFields.length; i++) {
            var field = localFields[i];
            // field refers to a DataSource field; stop
            if (field.name != null && parentFields[field.name] != null) return;

            if (shouldUseFieldFunc == null ||
                !shouldUseFieldFunc(field, this, true)) continue;

            // field doesn't refer to a DataSource field; add it and continue
            // NOTE: type defaults are auto-applied to DS fields and combined fields
            isc.SimpleType.addTypeDefaults(field);
            combinedFields.add(field);
        }
    },

    //> @method combineFieldData()  ([IA])
    //  Internal method to combine data from a field defined on the dataSource with a field object
    //  passed in.
    // @param   field   (object)    field to update with DS field defaults.
    // @param   targetFieldName   (string)    name of the DS field to copy properties from when
    //    the "field" object passed in the first param is not in the DS
    // @return  (object)            field with its data combined with the DS data
    // @visibility internal
    //<
    
    combineFieldData : function (localField, targetFieldName) {
        var parentField;
        if (isc.isAn.Object(targetFieldName)) parentField = targetFieldName;
        else parentField = this.getField(targetFieldName || localField.name);
        return isc.DataSource.combineFieldData(localField, parentField);
    },

    // SimpleType handling: local types and type defaults
	// ----------------------------------------------------------------------------------------

    // add validators and other defaults that can be declared as part of a type to the field
    // definition.  NOTE: DataSources do this once on init.  fields that don't come from a
    // DataSource do this during DataBinding (see Canvas.bindtoDataSource)
    _addTypeDefaults : function (fields) {
        
        if (fields == null) fields = this.fields;

        // add validators to each field, based on the specified type for the field
        for (var fieldName in fields) {
            var field = fields[fieldName];
            
            if (field && field.required == null && field.xmlRequired != null && 
                field.xmlNonEmpty != null) 
            {
                field.required = field.xmlRequired && field.xmlNonEmpty
            }

            // if a field is marked as the childrenProperty
            if (field && (field.childrenProperty || field.name == this.childrenField)) {
                            
                // for correct recursive transform of either XML or JSON, it needs to have the
                // type of this DataSource.
                if (!field.type) field.type = this.ID;
            }
            isc.SimpleType.addTypeDefaults(field, this);
            this._addFieldValidators(field);
        }
    },
        
    // Add validators that replace basic field properties (ex. required)
    _addFieldValidators : function (field) {

        var requiredValidator = {type: "required"};

        if (field.required) {
            var validator = isc.addProperties({}, requiredValidator),
                message = field.requiredMessage || this.requiredMessage
            ;
            if (message != null) validator.errorMessage = message;

            // Add validator to field
            if (!field.validators) {
                field.validators = [validator];
            } else {
                if (!isc.isAn.Array(field.validators)) {
                    field.validators = [field.validators];
                }
                // if the field is using the shared, default validators for the type, 
                // make a copy before modifying
                if (field.validators._typeValidators) {
                    field.validators = field.validators.duplicate();
                }
                field.validators.add(validator);
            }
        }
    },

    _autoDeriveTitles : function () {
        if (!this.autoDeriveTitles) return;
        for (var fieldName in this.fields) {
            var field = this.fields[fieldName];
            if (field.title != null) continue;

            field.title = this.getAutoTitle(fieldName);
            field._titleAutoDerived = true;
        }
    }, 

    //> @method dataSource.getAutoTitle()
    // Return a reasonable user-visible title given a fieldName.  Called when
    // +link{autoDeriveTitles} is true and by default, calls the class method
    // +link{classMethod:DataSource.getAutoTitle()}.  Override to provide a different policy
    // for auto-deriving titles for a particular DataSource or subclass of DataSource.
    //
    // @param identifier (String) identifier for which a title is desired.
    // @return (String) auto-derived title
    //
    // @group title
    // @visibility external
    //<
    getAutoTitle : function (identifier) {
        return isc.DataSource.getAutoTitle(identifier);
    },

    // get the simple type object for the specified typeName, if any, taking into account types
    // declared just on this DataSource
    _$type: "type",
    getType : function (typeName) {
        if (this.schemaNamespace) {
            var schemaSet = isc.SchemaSet.get(this.schemaNamespace),
                type = schemaSet.getSimpleType(typeName);
            if (type) return type;
        }

        // search for globally defined type first
        var type = isc.SimpleType.getType(typeName);
        if (type != null) return type;
        // then search for locally defined types
        if (this.types && this.types[typeName]) return this.types[typeName];
        return null;
    },
    
    //>LocalDS mode (serverless DataSource)
	// -----------------------------------------------------------------------------------------


    firstCacheAllDataRequest : function (dsRequest) {

        if (this.logIsInfoEnabled("cacheAllData")) {
            this.logInfo("firstCacheAllDataRequest: refreshing cache", "cacheAllData");
        }
        // defer all other operations against this datasource until this fetch
        // completes, starting with this one
        this._deferredRequests = [dsRequest];

        // In cacheAllData mode, the dataSource becomes clientOnly wrt fetches until
        // an explicit refresh of the cache is issued.
        // As such standard 'transformRequest' / 'transformResponse' logic is likely inapplicable.
        // We can't simply throw away transformRequest - it will still need to be called for
        // non fetch operations and for fetches if the cache is refreshed. Instead
        // wrap it in a function that no-op's if the request is clientOnly
        if (this._insulatedTransforms == null) {
        	// Rename original transform methods
        	// Note: If these are being observed, be sure to copy across the underlying
        	// method, not the observation notification method
        	var transformRequest = this.transformRequest,
                transformResponse = this.transformResponse,
                observers = this._observers;
            if (observers) {
                if (observers.transformRequest) {
                    transformRequest = this[isc._obsPrefix + "transformRequest"]
				}
				if (observers.transformResponse) {
					transformResponse = this[isc._obsPrefix + "transformResponse"]
				}
			}
			this.transformServerRequest = transformRequest;
			this.transformServerResponse = transformResponse;
			
			// Replace with functions that call the original logic if the request/response isn't
			// marked as clientOnly
			// Note that addMethods will preserve observation.
            // dataProtocol: clientCustom is an odd case, we need to call the original
            // transformRequest (now called transformServerRequest) or we'll
            // never receive a response at all.
            this.addMethods({
                _isServerRequest : function (dsRequest) {
                    
                    return dsRequest.cachingAllData ||
                        (this.cacheAcrossOperationIds && 
                            (dsRequest.operationType && dsRequest.operationType != "fetch")) ||
                        (this.cacheAcrossOperationIds === false && 
                            this.cacheAllOperationId != 
                                (dsRequest.operationId || dsRequest.operation)
                        )
                    ;
                },
                transformRequest : function (dsRequest) {
                    if (!this._isServerRequest(dsRequest)) return dsRequest;
                    return this.transformServerRequest(dsRequest);
                },
                transformResponse : function (dsResponse,dsRequest,data) {
                    if (!this._isServerRequest(dsRequest)) {
                        var cacheAllDataTime = this._autoCacheAllData_timestamp,
                            sentTime = dsRequest._sentTime;

                        if (!cacheAllDataTime || !sentTime || sentTime >= cacheAllDataTime) 
                            return dsResponse;
                    }
                    return this.transformServerResponse(dsResponse,dsRequest,data);
                }
            });
			
			// Set the flag to indicate we've set up these 'insulated' methods
			
			this._insulatedTransforms = true;
        }

        this.cacheResultSet = isc.ResultSet.create({
            dataSource: this,
            fetchMode: "local",
            fetchOperation: this.cacheAcrossOperationIds? dsRequest.operationId: this.cacheAllOperationId,
            allRows: this.cacheData ? this.cacheData : null,
            cachingAllData: true,
            componentId : "(cacheAllData fetch)",
            componentContext : "(from: " + dsRequest.componentId + 
                    (dsRequest.componentContext ? " &lt;" + dsRequest.componentContext + "&gt;)" 
                                                : ")"),
            dataArrived : function (startRow, endRow) {
                if (this.logIsInfoEnabled("cacheAllData")) {
                    this.logInfo("cacheAllData - cacheResultSet.dataArrived: startRow/endRow: "+startRow+"/"+endRow);
                }
                if (this.lengthIsKnown()) {
                    var ds = this.getDataSource();
                    if (ds.cacheResultSet == null) return;
                    ds.cacheLastFetchTime = new Date().getTime();
                    // if both cacheAllData and clientOnly are set, we do the initial fetch
                    // here according to cacheAllData, but we then switch to clientOnly by 
                    // setting ds.testData/cacheData
                    if (ds.clientOnly) ds.testData = ds.cacheData = this.getAllRows();
                    
                    ds.processDeferredRequests();
                }
            }
        });

        if (!this.cacheData) {
            if (this.logIsInfoEnabled("cacheAllData")) {
                this.logInfo("firstCacheAllDataRequest: issuing fetch", "cacheAllData");
            }
            this.cacheResultSet.get(0);
            return true; // request has been deferred, return true to halt further processing
        } else {
            if (this.logIsInfoEnabled("cacheAllData")) {
                this.logInfo("firstCacheAllDataRequest: updating last fetch time", "cacheAllData");
            }
            this.cacheLastFetchTime = new Date().getTime();
            if (this.clientOnly) this.testData = this.cacheData;
            this.processDeferredRequests();
            // request was deferred and has just been processed - return true to halt further 
            // processing
            return true;
        }

    },
    
    //> @method dataSource.getCacheData()
    // Returns the complete set of data cached by this dataSource. Note that this may 
    // have been supplied via +link{dataSource.cacheData}, or may have been fetched
    // from the server for dataSources with +link{dataSource.cacheAllData} set to true.
    // @return (Array of Records) entire cached set of data
    // @visibility external
    //<
    getCacheData : function () {
    	// cacheAllData RS's use "cacheResultSet" rather than just cacheData.
    	if (this.cacheResultSet != null) {
    		if (this.cacheResultSet.allRows) return this.cacheResultSet.allRows;	
    	}
    	if (this.cacheData != null) {
    		return this.cacheData;
    	}
    	if (this.testData != null) {
    		return this.testData;
    	}
    },

	// Given a dsRequest - if this is a clientOnly / cacheAllData dataSource, do we currently
	// have a cache to perform client-side operations against?
	// Returns false if the cache exists allowing the calling code in sendDSRequest to continue
	// and perform a client-side fetch against that test data.
	// Otherwise kicks off the one-time fetch if necessary, adds the request to the 
	// deferredRequests queue and returns true indicating that the calling code should stop
	// processing and wait for the one time fetch to return.
    fetchingClientOnlyData : function (dsRequest) 
    { 
        if (dsRequest.cachingAllData) {
            //delete dsRequest.cachingAllData;
            return false;
        }
        
        // when downloadResult is true, we bypass the cache and hit the server
        // even if dataSource.cacheAllData is true
        if (dsRequest.downloadResult) {
        	return false;
        }

        var useTestDataFetch = (this.useTestDataFetch == null ? 
                                this.shouldUseTestDataFetch() :
                                this.useTestDataFetch);

        if (this.logIsInfoEnabled("cacheAllData")) {
            this.logInfo("fetchingClientOnlyData: useTestDataFetch is "+useTestDataFetch, "cacheAllData");
        }

        // mark the request client-only: this causes the RPCManager to avoid sending
        // this RPC to the server.  Note that, if clientOnly requests were not allowed to be
        // involved in transactions, we could just fire the passed-in callback immediately with
        // the client-only dsResult
        if (this.clientOnly) {
            dsRequest.clientOnly = true;
            if (this.testData && !this.cacheData) this.cacheData = this.testData;
            else if (this.cacheData && !this.testData) this.testData = this.cacheData;
        }

        // if we're deferring requests, add request to queue and return
        if (this._deferredRequests) {
            this._deferredRequests.add(dsRequest);
            return true; // request has been deferred, return true to halt further processing
        }
        
        var cacheNeedsRefresh = this.cacheNeedsRefresh();
        
        if (useTestDataFetch == false && this.clientOnly && this.hasTestData()) 
            useTestDataFetch = true;
        if (!useTestDataFetch && ((this.cacheAllData && cacheNeedsRefresh) ||
                (this.clientOnly && !this.testData && this.hasTestData())
            ))
        {
            // we're not using a testDataFetch and either we're in cacheAllData mode and the
            // cache needs refreshing, or we're in clientOnly mode and the cache hasn't been 
            // fetched yet
            return this.firstCacheAllDataRequest(dsRequest);
        } else {
            // if we're in clientOnly or cacheAllData mode and there's no testData, but we do 
            // have a dataURL or testFileName, do a one-time load, then re-run current 
            // operation against the resulting testData.  If we're in cacheAllData mode, 
            // initialize the cacheResultSet with the resulting testData - further client-side
            // fetching will take place against that resultSet.
            if (this.clientOnly && !this.testData && (this.testFileName || this.dataURL) ||
                (this.cacheAllData && cacheNeedsRefresh)){

                if (this.logIsInfoEnabled("cacheAllData")) {
                    this.logInfo("fetchingClientOnlyData: issuing oneTimeDS fetch", "cacheAllData");
                }

                // defer all other operations against this datasource until this fetch
                // completes, starting with this one
                this._deferredRequests = [dsRequest];

                // perform a one-time fetch by creating a datasource that picks up the fields,
                // but not the operationBindings or other properties.  This way we don't have
                // to muck with this DS and try to restore settings

                // if only testFileName is set, use that as the dataURL
                var dataURL = this.dataURL || this.testFileName;

                // if dataFormat is the default iscServer, set it based on the file extension
                // of the dataURL, since it doesn't really make sense to have a clientOnly
                // dataSource of type iscServer
                var dataFormat = this.getDataFormat(dsRequest);
                if (dataFormat == "iscServer") dataFormat = dataURL.match(/\.xml$/i) ? "xml" : "json";

                var operationBinding = this.getOperationBinding(dsRequest);

                // Note: if someone's observing transformRequest/response, ensure we copy the
                // original methods across - otherwise we'll get JS errors since we're not also
                // copying the observers across to this one-time DS.
                
                var transformRequest = this.transformRequest,
                    transformResponse = this.transformResponse,
                    observers = this._observers;
                if (observers) {
                    if (observers.transformRequest) {
                        transformRequest = this[isc._obsPrefix + "transformRequest"]
                    }
                    if (observers.transformResponse) {
                        transformResponse = this[isc._obsPrefix + "transformResponse"]
                    }
                }
                // check for recordName on the opBinding, then on this DS - failing 
                // that, if this DS inherits from another DS, use the ID of the parent before 
                // the child to work out the recordName
                var localRecordName = operationBinding.recordName || this.recordName ||
                        (this.inheritsFrom ? (
                            isc.isA.String(this.inheritsFrom) ? 
                                this.inheritsFrom : this.inheritsFrom.ID) 
                        : this.ID);
                var oneTimeDS = isc[this.Class].create({
                    ID: this.ID+"_oneTime",
                    inheritsFrom: this.ID,
                    dataURL: dataURL,
                    dataFormat: dataFormat,
                    recordXPath: this.recordXPath,
                    // use our transformRequest/response on the one-time DS to canonicalize data to
                    // our internal testData format.
                    transformRequest: transformRequest,
                    transformResponse: transformResponse,
                    recordName: localRecordName,
                    showPrompt: this.showPrompt
                });
                
                this.logInfo("clientOnly datasource performing one-time " + dataFormat +
                             " fetch via: " + dataURL);

                // reset our transformRequest/response to default DataSource versions b/c we'll
                // be working against a local testData from now on and transforms no longer apply.
                this.addProperties({
                    transformRequest: isc.DataSource.getInstanceProperty("transformRequest"),
                    transformResponse: isc.DataSource.getInstanceProperty("transformResponse")
                });

                var ds = this;
                // note: don't pass criteria to the one-time fetch as we want to fetch all the
                // records for our cache, we'll apply the criteria in a follow-up local
                // operation against the cache 

                if (this.cacheAllData) {
                    oneTimeDS.cacheAllData = false;
                }

                oneTimeDS.sendDSRequest({
                    operationType : "fetch",
                    willHandleError:true,
                    // copy over the componentId and componentContext so it's easy to see what
                    // request initiated this one
                    componentId:dsRequest.componentId,
                    componentContext:dsRequest.componentContext,
                    callback : function (dsResponse, data) {
                        var cacheRows;
                        if (dsResponse.status != isc.DSResponse.STATUS_SUCCESS) {
                            ds.logWarn("one-time fetch failed with status: " + dsResponse.status +
                                       " and messsage: " + (data ? data : "N/A") + 
                                       ".  Initializing an empty Array as testData.");
                            cacheRows = [];
                        } else {
                            ds.logInfo("One-time fetch complete: "+ (data ? data.length : "null") + " records");
        
                            cacheRows = ds.initializeSequenceFields(data);
                        }

                        if (ds.cacheAllData) {
                            ds.cacheLastFetchTime = new Date().getTime();
                            ds.cacheResultSet = isc.ResultSet.create({
                                dataSource: ds.ID,
                                fetchMode: "local",
                                allRows: cacheRows
                            });
                            ds.cacheLastFetchTime = new Date().getTime();
                        }

                        if (ds.clientOnly) {
                            ds.cacheData = ds.testData = cacheRows;
                        }

                        ds.processDeferredRequests();
                        oneTimeDS.destroy();
                    }
                });

                return true; // request has been deferred, return true to halt further processing
            }
        }
    },
    
    // NOTE: These methods are overridden in RestDataSource
    shouldUseTestDataFetch : function () {
        return this.clientOnly == true && 
               this.cacheAllData != true && 
               (this.dataURL != null || this.testFileName != null);
    },
    hasTestData : function () {
        return this.dataURL != null || this.testFileName != null;
    },

    //> @method dataSource.getClientOnlyResponse()
	// Return a "spoofed" response for a +link{clientOnly} or +link{cacheAllData} DataSource.
    // <P>
    // The default implementation will use +link{dataSource.cacheData} to provide an appropriate
    // response, by using +link{applyFilter,client-side filtering} for a "fetch" request, and
    // by modifying the <code>testData</code> for other requests.
    // <P>
    // Override this method to provide simulations of other server-side behavior, such as
    // modifying other records, or to implement <b>synchronous</b> client-side data providers
    // (such as Google Gears).  For <b>asynchronous</b> third-party data providers, such as
    // GWT-RPC, HTML5 sockets, or bridges to plug-in based protocols (Java, Flash,
    // Silverlight..), use +link{DSProtocol,dataProtocol:"clientCustom"} instead. 
    // <P>
    // Overriding this method is also a means of detecting that a normal DataSource (not
    // clientOnly) would be contacting the server.
    //
    // @param request (DSRequest) DataSource request to respond to
    // @param serverData (Array of Record) for cacheAllData DataSources, the data from the local cache
    // @return (DSResponse) 
    // @visibility external
    //<
	getClientOnlyResponse : function (request, serverData) {
        //!OBFUSCATEOK
        // initialize the spoofed dataset

        serverData = serverData || this.testData;

        // if we have serverData but no testData, it's a cacheAllData result - if we're also in
        // clientOnly mode, we need to set up the clientOnly cache - set both cacheData and the
        // now deprecated testData to the same array
        if (serverData && !this.testData && this.clientOnly) 
            this.cacheData = this.testData = serverData;

        if (!serverData || isc.isA.String(serverData)) {
		    if (isc.isA.String(serverData)) {
                // ID or expression
                //>DEBUG
				this.logInfo(this.ID + " datasource: using testData property as data");
                //<DEBUG
                this.cacheData = this.testData = isc.eval(serverData);
			} else if (window[this.ID + "TestData"]) {
                // dataset loaded in page under canonical name
                //>DEBUG
				this.logInfo(this.ID + " datasource: using " + this.ID + "TestData object as data");
                //<DEBUG
				this.cacheData = this.testData = window[this.ID + "TestData"];
            } else {
                // initialize to empty list
                //>DEBUG
				this.logInfo(this.ID + " datasource: testData property and " + this.ID +
							 "TestData object not found, using empty list as data");
                //<DEBUG
				this.cacheData = this.testData = [];
			}
            serverData = this.testData;
        }

		var operationType = request.operationType,
			response = {
				status:0
			};

		switch (operationType) {
			case "fetch":
			case "select":
			case "filter":
                var criteria = request.data;
                // support multiple where clause format (don't think this is actually valid any
                // more)
                if (isc.isAn.Array(criteria)) criteria = criteria[0];
                var filteredData = this.applyFilter(serverData, criteria, request),
                    resultData = filteredData;

                if (request.startRow != null) {
                    // simulate paging
                    var startRow = request.startRow,
                        endRow = request.endRow,
                        totalRows = filteredData.length;

                    // server handles sort, so sort the data by the requested direction
                    
                    var sortProperty = isc.shallowClone(request.sortBy);
                    if (sortProperty) {
                        if (!isc.isAn.Array(sortProperty)) sortProperty = [sortProperty];

                        if (isc.isAn.Object(sortProperty[0])) {
                            // if this is an array or SortSpecifiers, map them to sortBy strings
                            sortProperty = isc.DS.getSortBy(sortProperty)
                        }

                        var sortDirs = [];
                    
                        for (var i = 0; i < sortProperty.length; i++) {
                            var sortDirection = true; 
                            if (sortProperty[i].startsWith("-")) {
                                sortProperty[i] = sortProperty[i].substring(1);
                                sortDirection = false;
                            }
                            sortDirs[i] = sortDirection;
                        } 
                        filteredData.sortByProperties(sortProperty, sortDirs);
                    }

                    // cap the endRow at one after last valid index (it's exclusive)
                    endRow = Math.min(endRow, totalRows);
                    // slice out from startRow to endRow (note: slice is non-inclusive at the end)
				    resultData = filteredData.slice(startRow, endRow);
                    response.startRow = startRow;
                    response.endRow = endRow;
                    response.totalRows = totalRows;
                }

                // shallow copy the results.  This allows test or example cases where the
                // "server data set" is changing independently of the client results
                if (this.copyLocalResults) {
                    var isSGWT = isc.Browser.isSGWT;
                    for (var i = 0; i < resultData.length; i++) {
                    	if (this.deepCopyLocalResults) {
							resultData[i] = isc.clone(resultData[i]);
	                    } else {
	                        resultData[i] = isc.addProperties({}, resultData[i]);
						}
                        // Clear out SGWT's __ref/__module pointers so that if the cacheData is created in
                        // SGWT, Record.getOrCreateRef() won't return the same Java Record object for the
                        // copied result record.
                        if (isSGWT) {
                            resultData[i][isc.gwtRef] = null;
                            resultData[i][isc.gwtModule] = null;
                        }
                    }
                }

                response.data = resultData;
				break;
            // NOTE: for modification operations, we always return a copy, "copyLocalResults"
            // doesn't need to be set.
			case "remove":
			case "delete": // old name
                if (this.isMissingPrimaryKeys(request.data)) {
                    response.data = "clientOnly remove operation failed for DataSource " + 
                                        this.ID + ": missing primaryKey values(s) " +
                                        this.getMissingPrimaryKeys(request.data);
                    response.data = "clientOnly remove operation failed: missing primaryKey " +
                                    "value(s): " + this.getMissingPrimaryKeys(request.data);
                    response.status = -1;
                } else {
                    var serverRecordIndex = this.findByKeys(request.data, serverData);
                    if (serverRecordIndex == -1) {
                        this.logWarn("clientOnly remove operation: Unable to find record matching criteria:" 
                                    + this.echo(request.data));
                        response.data = "clientOnly remove operation failed: unable to find matching " +
                                        "record"
                        response.status = -1;
                    } else {
                        serverData.removeAt(serverRecordIndex);
                        response.data = isc.addProperties({}, request.data);
                    }
                }
				break;
			case "add": 
			case "insert": // old name
                // Generate values for sequence fields at this point
                var serverRecord = isc.addProperties({}, request.data);
                
                if (serverRecord.__ref) {
                    delete serverRecord.__ref;
                    delete serverRecord.__module;
                }
                serverRecord = this.applySequenceFields(serverRecord);

                // Check for duplicate key
                var serverRecordIndex = this.findByKeys(serverRecord, serverData);
                if (serverRecordIndex != -1) {
                    this.logWarn("clientOnly add operation: Duplicate key: " +
                                        isc.echoAll(serverRecord));
                    response.data = "clientOnly add operation failed for DataSource " + 
                                        this.ID + ": Duplicate key in record " +
                                        isc.echoAll(serverRecord);
                    response.status = -1;
                } else {
                    // make both the saved data and returned data a distinct copy 
                    // from the passed data
                    serverData.add(serverRecord);
                    response.data = isc.addProperties({}, serverRecord);
                }
				break;
			case "replace":
			case "update":
                if (this.isMissingPrimaryKeys(request.data)) {
                    this.logWarn("clientOnly update operation: Missing primaryKey values: " +
                                        this.getMissingPrimaryKeys(request.data));
                    response.data = "clientOnly update operation failed for DataSource " + 
                                        this.ID + ": missing primaryKey values(s) " +
                                        this.getMissingPrimaryKeys(request.data);
                    response.status = -1;
                } else {
                    // find the server record
                    var serverRecordIndex = this.findByKeys(request.data, serverData);
                    if (serverRecordIndex == -1) {
                        this.logWarn("clientOnly update operation: Unable to find record matching criteria:" 
                                    + this.echo(request.data));
                        response.data = "clientOnly update operation failed: unable to find matching " +
                                        "record. Did you supply all primaryKeys?"
                        response.status = -1;
                    } else {
                        var serverRecord = serverData[serverRecordIndex];
                        // update the server record in place
                        for (var key in request.data) {

                            if (key == "__ref" || key == "__module") continue;

                            var field = this.getField(key);
                            // Note we are not passing in the 'field' object as a parameter.
                            // This is because if the field is of a simpleType with
                            // a get/update atomic value pair we do not want to run through
                            // the 'updateAtomicValue' method -- the record passed in should
                            // already have the field value in opaque format.                     
                            isc.Canvas._saveFieldValue(key, null, request.data[key], serverRecord, null, true);
                        }
                        // return a copy
                        response.data = isc.addProperties({}, serverRecord);
                    }
                }
				break;
			case "validate":
			default:
				break;
		}

        this._trackClientOnlyChanges(response, operationType);

		return response;
	},

    _trackClientOnlyChanges : function(response, operationType) {

        // we will store the changes in separate arrays, according to operationType
        // storing the response data, and the arrays will be indexed by primary key value
        var pk = this.getPrimaryKeyFieldName();

        switch(operationType) {
            case 'add':
                if (!this._addedRecords) {
                    this._addedRecords = new Array();
                }
                this._addedRecords[response.data[pk]] = response.data;
                break;

            case 'update':
                if (!this._updatedRecords) {
                    this._updatedRecords = new Array();
                }

                // if we're trying to update a previously added record, then update that 
                // record's values instead of recording an update
                if (this._addedRecords && this._addedRecords[response.data[pk]]) {
                    this._addedRecords[response.data[pk]] = response.data;
                } else {
                    this._updatedRecords[response.data[pk]] = response.data;
                }
                break;

            case 'remove':
                if (!this._removedRecords) {
                    this._removedRecords = new Array();
                }
                
                // if a newly added record is removed, then should not be added at all
                // so we remove it from the list of added records
                if (this._addedRecords && this._addedRecords[response.data[pk]]) {
                    this._addedRecords[response.data[pk]] = null;
                }

                // if an updated record is removed, then should not be updated
                // so ve remove it from the list of updated records
                if (this._updatedRecords && this._updatedRecords[response.data[pk]]) {
                    this._updatedRecords[response.data[pk]] = null;
                }
                this._removedRecords[response.data[pk]] = response.data;
        }
    },

    //> @method dataSource.getChanges()
    //
    // Return the changes made on a client-only DataSource in a form of DSRequest array
    // which can be used to update the records with a non-client only DataSource.
    // <p>
    // The list is built according to following rules:
    // <ul>
    //  <li>Updates to the same record are collapsed into a single update.</li>
    //  <li>Deletions of a record cancels any previous updates to the record.</li>
    //  <li>If a new record is created and then updated, this results in one "add" operation using the latest values.</li>
    //  <li>If a new record is removed, there will be no requests for it.</li>
    // </ul>
    //
    // @return (Array of DSRequest) request objects containing the changes on this client-only DataSource.
    //<
    getChanges : function() {

        var result = new Array();
        var pk = this.getPrimaryKeyFieldName();

        if (this._addedRecords) {
            for(var id in this._addedRecords) {
                if (this._addedRecords[id] != null && isc.isA.Object(this._addedRecords[id])) {
                this._addedRecords[id][pk] = null;
                    var req = {
                        data: this._addedRecords[id],
                        operationType:'add'
                    }
                    result.push(req);
                }
            }
	    }

        if (this._updatedRecords) {
            for(var id in this._updatedRecords) {
                if (this._updatedRecords[id] != null && isc.isA.Object(this._updatedRecords[id])) {
                    var req = {
                        data : this._updatedRecords[id],
                        operationType:'update'
                    }
                    result.push(req);
                }
            }
        }

        if (this._removedRecords){
            for(var id in this._removedRecords) {
                if (this._removedRecords[id] != null && isc.isA.Object(this._removedRecords[id])) {
                    var req = {
                        data : this._removedRecords[id],
                        operationType:'remove'
                    }
                    result.push(req);
                }
            }
        }

        // remove the rows from the arrays used to track them, so at next call won't be reported again
        if (this._addedRecords) {this._addedRecords.splice(0, this._addedRecords.length);}
        if (this._updatedRecords) {this._updatedRecords.splice(0, this._updatedRecords.length);}
        if (this._removedRecords) {this._removedRecords.splice(0, this._removedRecords.length);}

        return result;
	},
    
    // Generating sequence values in client-only mode
    getNextSequenceValue : function (field) {
        var data = this.testData,
            value = 0;
        for (var i = 0; i < data.length; i++) {
            var fieldValue = data[i][field.name];
            if (fieldValue != null && fieldValue > value) value = fieldValue;
        }
        // So this returns the largest of the stored values, +1
        return value+1;
    },
    
    applySequenceFields : function (record) {

        if (!this.clientOnly) {
            
            return;
        }
        
        var fields = this.getFields();
        for (var fieldName in fields) {
            var field = fields[fieldName];
            // also auto-gen primaryKeys that may not be of type="sequence"
            
            if ((field.type == "sequence" || field.primaryKey) && record[fieldName] == null) {   
                var type = isc.SimpleType.getType(field.type);
                while (type.inheritsFrom) type = isc.SimpleType.getType(type.inheritsFrom);
                if (type == "date" || type == "time") {
                    this.logWarn("clientOnly dataSource encountered a primaryKey of date or " +
                                 "time type for which no value was supplied for an 'add' " +
                                 "request.  Defaulting to the epoch (midnight on 1 Jan 1970)");
                    record[fieldName] = new Date(0);
                } else {
                    record[fieldName] = this.getNextSequenceValue(field);
                    if (field.type != "sequence") {
                        this.logWarn("clientOnly dataSource found a missing value for primaryKey " +
                                     "field '" + fieldName + "' during an add request. Derived " +
                                     "value " + record[fieldName] + " from sequence processing");
                    }
                }
            }
        }
        return record;
    },
                      

    // generate primary keys in test data if not present (as frequently happens
    // where the storage layer is intended to auto-assign the primaryKey values
    // - e.g. Database sequences)
    //
    // This method assumes a potentially large number of records, where calling
    // applySequenceFields() per record would not be appropriate.  Part of the optimization
    // here is that we don't bother supporting gappy records - in other words, the primaryKeys
    // are either present or they're not - we don't attempt to intelligently fill them in as that
    // can be too slow
    initializeSequenceFields : function (records) {
        if (!isc.isAn.Array(records)) return;

        var fields = this.getFields();
        var sequenceFields = [];
        for (var fieldName in fields) {
            if (fields[fieldName].type == "sequence" || fields[fieldName].primaryKey) sequenceFields.add(fieldName);
        }

        for (var i = 0; i < records.length; i++) {
            for (var j = 0; j < sequenceFields.length; j++) {
                var fieldName = sequenceFields[j];
                if (records[i][fieldName] == null) records[i][fieldName] = i;
            }
        }
        return records;
    },
    
    isMissingPrimaryKeys : function(record) {
        return this.getMissingPrimaryKeys(record).length > 0;
    },
    
    getMissingPrimaryKeys : function(record) {
        var keys = this.getPrimaryKeyFieldNames(),
            missing = [],
            undef;
        if (!record) return keys;
        for (var i = 0; i < keys.length; i++) {
            if (record[keys[i]] == undef) {
                missing.add(keys[i]);
            }
        }
        return missing;
    },

    //<LocalDS
    
	// Local Filtering 
	// --------------------------------------------------------------------------------------------

    // given values for the primary key fields, find the index of the unique matching record
    findByKeys : function (record, recordSet, pos, endPos) {
    	if (record == null || recordSet == null) return -1;
    	
        var keys = this.getPrimaryKeyFieldNames(),
            filtered = {},
            dateKeys = false;
            
        // If we've been passed a ResultSet, the best we can do is search its current localData
        if (isc.ResultSet && isc.isA.ResultSet(recordSet)) recordSet = recordSet.localData;
        
        for (var i = 0; i < keys.length; i++) {
            filtered[keys[i]] = record[keys[i]];
            if (this.getField(keys[i]).type == "date" || this.getField(keys[i]).type == "time" ||
                this.getField(keys[i]).type == "datetime")
            {
                dateKeys = true;
            }
        }
        
        // Array.findByKeys() is much faster than DataSource.recordsMatchingFilter(), but it
        // doesn't work with key fields of date type
        if (!dateKeys) return recordSet.findByKeys(record, this, pos, endPos);
        
        var matching = this.recordIndicesMatchingFilter(recordSet, filtered, {}, pos, endPos);
        if (!matching || matching.length == 0) return -1;
        if (matching.length > 1) {
            this.logWarn("Searching by primaryKey, found " + matching.length + " records " +
                         "when there should only be one - returning the first.  PrimaryKey " +
                         "set was: " + isc.echoAll(filtered));
        }
        return matching[0];
    },

    //> @method dataSource.applyFilter()
    // Returns records in the passed Array that match the provided filter
    // +link{type:Criteria,criteria}. Handles simple or +link{AdvancedCriteria,advanced} criteria.
    // <P>
    // By default:
    // <ul>
    // <li> any criteria that do not correspond to a DataSource field are ignored
    // <li> for simple criteria, any null or empty string criteria are ignored and
    //      all other criteria are passed to +link{fieldMatchesFilter()}
    // <li> for advanced criteria, each criterion is evaluated via +link{evaluateCriterion()}
    // </ul>
    // This method is called by +link{resultSet.applyFilter()} to provide filtering when a
    // ResultSet has a complete cache and filtering can be performed client-side.  You may want
    // to override this method in order to mimic the filtering behavior that your server performs.
    //
    // @param   data        (Array)     the list of rows
    // @param   criteria    (Criteria)  the filter criteria
    // @param   [requestProperties]   (DSRequest Properties)  optional dataSource request properties
    // @return  (Array)     the list of matching rows
    // @visibility external
    //<
	applyFilter : function (data, criteria, requestProperties) {
		var output = [];
		if (!data || data.length == 0) return output;

        // If our criteria object is of type AdvancedCriteria, go down the new
        // AdvancedFilter codepath
        if (this.isAdvancedCriteria(criteria)) {
            var normalizedCriteria = this.normalizeAdvancedCriteria(criteria);
            return this.recordsMatchingAdvancedFilter(data, normalizedCriteria, requestProperties);
        }

        // go through the list of items and add any items that match the criteria to the
        // output
        return this.recordsMatchingFilter(data, criteria, requestProperties);
    },

	//> @attr dataSource.dropUnknownCriteria (Boolean : true : IR)
	// If the criteria applied to a fetch type operation contain fields that are not present
	// in the dataSource, should they be ignored when performing filtering on the client.
	// This property is useful for cases where you custom server logic makes use of criteria 
	// values to determine what set of records to return to the client, but the data
	// does not actually have record values for these fields and as such the client-side filtering
	// logic should ignore them.
	// @visibility external
	//<
	dropUnknownCriteria:true,

    // return the indices of the records matching the criteria passed in
    recordIndicesMatchingFilter : function (records, filter, requestProperties, startPos, endPos) {
        return this.recordsMatchingFilter(records, filter, requestProperties, startPos, endPos, true);
    },

    // return the records matching the criteria passed in
    recordsMatchingFilter : function (records, filter, requestProperties, startPos, endPos, returnIndices) {
        var filterFields = isc.getKeys(filter),
            filterFieldsLength = filterFields.length,
            matches = [],
            record,
            match,
            fieldName,
            fieldValue,
            filterValue,
            j;
            
        startPos = startPos || 0;
        endPos = endPos || records.length;
        
        // Get any customCriteriaField definition from the operationBinding for later checking
        if (requestProperties && requestProperties.operation && this.operationBindings) {

            var op = requestProperties.operation;
            if (op.ID == op.dataSource + "_" + op.type) {
                var opBinding = this.operationBindings.find({operationId:null, operationType:op.type});
            } else {
                var opBinding = this.operationBindings.find({
                    operationId: requestProperties.operation.ID,
                    operationType: op.type
                });
            }

            if (opBinding) {
                var customCriteriaFields = opBinding.customCriteriaFields;
                if (isc.isA.String(customCriteriaFields)) {
                    customCriteriaFields = customCriteriaFields.split(",");
                    // Better trim them...
                    for (var k = 0; k < customCriteriaFields.length; k++) {
                        customCriteriaFields[k] = customCriteriaFields[k].replace(/^\s+|\s+$/g, '');
                    }
                }
            }
        }

        for (var i = startPos; i < endPos; i++) {
            record = records[i];
            if (record == null) continue;
            match = true;

            for (j = 0; j < filterFieldsLength; j++) {
                fieldName = filterFields[j];
                if (fieldName == null) continue; // null property in criteria
                // Pass in the param to handle being passed a dataPath
                var field = this.getField(fieldName, true);
                if (field == null) field = this.getFieldForDataPath(fieldName);
                // Ignore fields that aren't present in the DataSource.  This allows criteria
                // unrelated to the fields to be present but not interpreted by DataSource
                // filtering (eg, possibly only interpreted by server filtering)
                // Exception - if a dataPath is specified that leads to a nested field, 
                // allow filtering to proceed (we'll use the dataPath to extract the value from the
                // record directly).
                if (this.dropUnknownCriteria && !field) {
                	continue;
                }

                // Obey the DataSourceField.customSQL and OperationBinding.customCriteriaFields
                // properties
                var skipCustomSQLCheck = false;
                if (isc.isA.List(customCriteriaFields) && customCriteriaFields.contains(fieldName)) {
                    skipCustomSQLCheck = true;
                }
                
                if (!skipCustomSQLCheck && field && field.customSQL) continue;
                
                fieldValue = isc.DataSource.getPathValue(record, fieldName, field);
                filterValue = filter[fieldName];
                if (!this.fieldMatchesFilter(fieldValue, filterValue, requestProperties, 
                     (field ? field.ignoreTextMatchStyle : null))) 
                {
                    match = false;
                    break;
                }
            }
            if (match) matches.add(returnIndices ? i : record);
		}
		return matches;
	},

	recordMatchesFilter : function (record, criteria, requestProperties) {
		if (this.isAdvancedCriteria(criteria)) {
		    return this.recordsMatchingAdvancedFilter([record], criteria, requestProperties).length > 0;
		}
        return this.recordsMatchingFilter([record], criteria, requestProperties).length > 0;
    },

    //> @method dataSource.fieldMatchesFilter()
    // Compares a criteria value to a field value and returns whether they match, as follows:
    // <ul>
    // <li> any non-String filter value is directly compared (==) to the field value
    // <li> any String filter value is compared according to
    //      +link{dsRequest.textMatchStyle} in the passed <code>requestProperties</code>,
    //      regardless of the actual field type
    // <li> if the filter value is an Array, the comparison is a logical OR.  If textMatchStyle
    //      is "exact", it matches if fieldValue (or any of it's entries, if it's also an array) 
    //      is contained in the filterValue Array.  If textMatchStyle if substring, it matches 
    //      if any of the entries in filterValue appear as a case-insensitive substring of any
    //      of the entries in fieldValue.
    // </ul>
    // @param   fieldValue  (object)    field value to be compared
    // @param   filterValue (object)    filter value to be compared
    // @param   [requestProperties]   (DSRequest Properties)  optional dataSource request properties 
    // @return  (boolean)               true if the filter and field values match, false otherwise
    // @visibility external
    //<
    _$startsWith : "startsWith", _$substring : "substring", _$exact:"exact",
    fieldMatchesFilter : function (fieldValue, filterValue, requestProperties, ignoreTextMatchStyle) {
        // Pick up textMatchStyle from the request, if present
        var textMatchStyle;
		if (ignoreTextMatchStyle) {
            textMatchStyle = this._$exact;
        } else if (requestProperties) {
            textMatchStyle = requestProperties.textMatchStyle;
        }

        if (isc.isAn.Array(filterValue) || isc.isAn.Array(fieldValue)) {
            // filterValue is an array - always treat this as a logical OR, and match either 
            // exact or substring, according to textMatchStyle
            if (!isc.isAn.Array(fieldValue)) {
                fieldValue = [fieldValue];
            }

            if (!isc.isAn.Array(filterValue)) {
                filterValue = [filterValue];
            }

            var isDate = isc.isA.Date(filterValue[0]);
            // dates only really work with exact match, so assume that if isDate = true
            if (textMatchStyle == this._$exact || isDate) {
                
                if (isDate) return filterValue.intersectDates(fieldValue).length > 0;
                else return filterValue.intersect(fieldValue).length > 0;
            } else {
                
                return filterValue.intersectSubstring(fieldValue).length > 0;
            }
        }
        // note: in general the fieldValues are part of records that came from the server and are
        // generally of the same type as the DataSource field.  The filterValues come from a form 
        // and are generally strings (3/25/02), even if the field is numeric.

		// handle multiple selects (array of filter values)
        

        if (isc.isA.Date(fieldValue) && isc.isA.Date(filterValue)) {
            if (filterValue.logicalDate) 
                return (Date.compareLogicalDates(fieldValue, filterValue) == 0);
            return (Date.compareDates(fieldValue, filterValue) == 0);
        }

		if (!isc.isA.String(fieldValue) && !isc.isA.String(filterValue)) {        
			// if both are non-string values, use direct comparison
            // XXX really, we need some way that you can supply a comparison function for comparing
            // two instances of a custom type.
            
			//>DEBUG
            if (this.logIsDebugEnabled()) {
                this.logDebug("Direct compare: " + fieldValue + "==" + filterValue);
            }
			//<DEBUG
			return (fieldValue == filterValue); 
		}
        // Convert nulls to emptyStrings
        
        if (filterValue == null) filterValue = isc.emptyString;
        if (fieldValue == null) fieldValue = isc.emptyString;
			
        // if the field is a string or is numeric, we want to do substring match.  Thus "part"
        // matches "part number" and "17" matches "017395".
        
        // convert both field and filter value to strings (note: they need to be strings because we
        // actually call string methods on them)
        if (!isc.isA.String(fieldValue)) fieldValue = fieldValue.toString();
        if (!isc.isA.String(filterValue)) filterValue = filterValue.toString();
        if (!this.filterIsCaseSensitive) {
            fieldValue = fieldValue.toLocaleLowerCase();
            filterValue = filterValue.toLocaleLowerCase();
        }

        // Normalize unsupported text match style to the default
        if (!this.supportsTextMatchStyle(textMatchStyle)) {
            // Only warn once for each text match style for this dataSource
            if (!this._warnedUnsupportedTMS) this._warnedUnsupportedTMS = {};
            if (!this._warnedUnsupportedTMS[textMatchStyle]) {
                this.logWarn("Text match style specified as '" + textMatchStyle + 
                             "': This is not supported for" +
                             " this dataSource - performing a substring match instead");
                this._warnedUnsupportedTMS[textMatchStyle] = true;
            }
            textMatchStyle = this.getTextMatchStyle(textMatchStyle);
        }

        if (textMatchStyle == this._$startsWith) {
            return isc.startsWith(fieldValue, filterValue);
        } else if (textMatchStyle == this._$substring) {        
            return isc.contains(fieldValue, filterValue); 
        // Default to exact match if textMatchStyle is unset. This matches standard fetch
        // behavior
        } else {
            return fieldValue == filterValue;
        }
	},
    
    _$iscServer:"iscServer",
    //> @method  dataSource.supportsTextMatchStyle() (A)
    // Does this dataSource support the specified "textMatchStyle" when performing a filter
    // operation against a text field.
    // @param textMatchStyle (TextMatchStyle) textMatchStyle to check. If passed a null value,
    //      assume an exact match is being requested.
    // @visibility external
    //<
    supportsTextMatchStyle : function (style, fetchMode) {
        // Assume completely custom DataSources can handle any textMatchStyle, as they can
        // be completely custom coded
        if (!this.clientOnly && (this.dataFormat != this._$iscServer)) return true;
        
        // return false if passed an unrecognized textMatchStyle string for standard 
        // ISCServer / client only dataSources
        return (style == null || style == this._$substring || style == this._$exact || 
                style == this._$startsWith);
    },

    // getTextMatchStyle()
    // Normalizes unsupported text match style to a default
    getTextMatchStyle : function (style) {
        
        if (style == null) style = this._$exact;
        if (!this.supportsTextMatchStyle(style)) {
            
            style = this._$substring;
        }
        return style;
    },
    
    // compareTextMatchStyle()
    // When changing to a new text match style, is the new style 'less restrictive'? 
    // Returns 0 if the text match style is unchanged,
    // -1 if the filter is less restrictive (So we would have to hit the server for new data
    // even if new criteria are unchanged or more restrictive)
    // 1 if the filter is more restrictive (meaning depending on the criteria we should be
    // able to perform a local filter of the client data when new criteria are set).
    compareTextMatchStyle : function (newStyle, oldStyle) {
        newStyle = this.getTextMatchStyle(newStyle);
        oldStyle = this.getTextMatchStyle(oldStyle);

        // Note: If new and old style match we'll look at the criteria to determine 
        // whether a server fetch will be required.
        if (newStyle == oldStyle) return 0;
        
        // exact is the most restrictive, the startsWith (then substring)
        if (newStyle == this._$exact) return 1;
        if (oldStyle == this._$exact) return -1;
        if (newStyle == this._$startsWith) return 1;
        return -1;  
    },
    
    //> @method dataSource.compareCriteria()
    // Given two sets of criteria, determine whether they are equivalent, the new criteria is
    // guaranteed more restrictive, or the new criteria is not guaranteed more restrictive,
    // returning 0, 1 or -1 respectively.
    // <P>
    // Comparisons between +link{AdvancedCriteria} are made via recursively calling
    // +link{Operator.compareCriteria()} for all criteria involved.
    // <P>
    // For simple +link{Criteria}, by default (+link{criteriaPolicy}:"dropOnShortening"), returns:
    // <ul>
    // <li> -1 if the new criteria has fewer properties than the old criteria (indicating that it
    //      isn't more restrictive)
    // <li> -1 if the value for any property in the old criteria is an array and 1) the value for
    //      the same property in the new criteria isn't an array, or 2) is an array but
    //      of different length, or 3) the arrays do not contain the exact same set of objects
    //      (order can be different)
    // <li> -1 if the value for any given property in the old criteria is not an array, and the
    //      the value for the same property property in the new criteria is different
    // <li> -1 if both values for a given property are strings and the new criteria value doesn't
    //      contain the old criteria value
    // <li> 1 if none of the above are true and, for at least one of the properties, the
    //      respective criteria values are both strings, and the old criteria value is a substring
    //      of, and is shorter than, the new criteria value
    // <li> 0 otherwise (indicating the sets of criteria are equivalent)
    // </ul>
    // <P>
    // For (+link{criteriaPolicy}:"dropOnChange"), returns:
    // <ul>
    // <li> -1 if the two sets of criteria have a different number of properties
    // <li> -1 if the value for any property in the old criteria is an array and 1) the value for
    //      the same property in the new criteria isn't an array, or 2) is an array but
    //      of different length, or 3) the arrays do not contain the exact same set of objects
    //      (order can be different)
    // <li> -1 if the value for any given property in the old criteria is not an array, and the
    //      the value for the same property in the new criteria is different
    // <li> 0 otherwise (indicating the sets of criteria are equivalent)
    // </ul>
    // This method is called by +link{resultSet.compareCriteria()} to determine whether a change
    // in criteria should cause the cache to be invalidated. You may want
    // to override this method in order to mimic the filtering behavior that your server performs.
    //
    // @param   newCriteria     (Criteria)  new filter criteria
    // @param   oldCriteria     (Criteria)  previous filter criteria
    // @param   [requestProperties]     (DSRequest Properties)  dataSource request properties
    // @param   [policy]        (string)    overrides +link{criteriaPolicy}
    // @return  (Number)    0 if the filters are equivalent, 1 if newCriteria is guaranteed more
    //                      restrictive, and -1 if newCriteria is not guaranteed more restrictive
    // @see criteriaPolicy
    // @visibility external
    //<
    compareCriteria : function (newCriteria, oldCriteria, requestProperties, policy) {
        //>DEBUG
        if (this.logIsInfoEnabled()) {
            this.logInfo("Comparing criteria, oldCriteria:\n" + this.echo(oldCriteria) +
                         "\nnewCriteria:\n" + this.echo(newCriteria) + 
                         ", policy: " + (policy || this.criteriaPolicy));
        }
        //<DEBUG

        if (oldCriteria == null) return -1;

        // get the textMatchStyle to be used with simple criteria
        var textMatchStyle = this.getTextMatchStyle(requestProperties ?
                                                    requestProperties.textMatchStyle : null);
		
        // If our criteria objects are of type AdvancedCriteria, go down the new
        // AdvancedFilter codepath
        if (this.isAdvancedCriteria(newCriteria) || this.isAdvancedCriteria(oldCriteria)) {
            var undef, 
                result;
                
            if (this.isAdvancedCriteria(newCriteria)) {
                if (this.isAdvancedCriteria(oldCriteria)) {
                    result = this.compareAdvancedCriteria(newCriteria, oldCriteria, 
                                                        requestProperties);
                } else {
                    // Special case: if the old criteria was the propertyless object, we're going from 
                    // no filter criteria to some filter criteria - so we know we're guaranteed to be more 
                    // restrictive (or at least, not to be less restrictive), so we can return 1.
                    var j = 0;
                    for (var i in oldCriteria) {
                    	j++; break;
					}
                    if (j == 0) result = 1; 
                }

                // We have a mix of basic and advanced criteria types - convert the basic criteria
                // object to an equivalent AdvancedCriteria.
                // (NOTE - this code is here as a catch-all only. ResultSet.setCriteria() explicitly 
                //  converts basic criteria to AdvancedCriteria as required, so we should never get as 
                //  far as this)
                if (result == undef) {
                    oldCriteria = isc.DataSource.convertCriteria(oldCriteria, textMatchStyle);
                    result = this.compareAdvancedCriteria(newCriteria, oldCriteria, 
                                                        requestProperties);
                }
            } else {
				// Special case: if the new criteria was the propertyless object, we're going from 
                // meaningful criteria to no filter criteria - so we know we're guaranteed to be less 
                // restrictive -1
                var j = 0;
                for (var i in newCriteria) {
                	j++; break;
				}           
                if (j == 0) {
                    result = -1; 
                } else {
                    // We have a mix of basic and advanced criteria types - convert the basic criteria
                    // object to an equivalent AdvancedCriteria
                    newCriteria = isc.DataSource.convertCriteria(newCriteria, textMatchStyle);
                    result = this.compareAdvancedCriteria(newCriteria, oldCriteria, 
                                                        requestProperties);
                }
            }
            // Catch-all - result should never be undefined at this point
            if (result == undef) result = -1;
            
            policy = policy || this.criteriaPolicy;
    
            if (policy == "dropOnShortening") {
                return result;
            } else {
                return result == 0 ? 0 : -1;
            }
        }
        
        policy = policy || this.criteriaPolicy;
    
        if (policy == "dropOnShortening") {
            // If we're doing an exact match, rather than a substring/startsWith match, 
            // any change to existing criteria will require a fresh set of data from the server.
            // (Note that introducing filtering to previously unrestricted fields will still be
            // more restrictive so won't require a fetch)
            if (textMatchStyle == this._$exact) {
                return this.dropOnFieldChange(newCriteria, oldCriteria, requestProperties);
            } else {
                return this.dropOnShortening(newCriteria, oldCriteria, requestProperties);
            }
        } else {
            return this.dropOnChange(newCriteria, oldCriteria, requestProperties);
        }
    },

    //> @method DataSource.getFieldCriterion
    // Returns the depth-first match of a criterion matching the given fieldName.
    //
    // @param criterion (Criteria) the criteria to search
    // @param fieldName (String) the fieldName to find criteria for
    // @return (Criteria) the depth-first matching criterion for the passed fieldName
    // @visibility external
    //<
    getFieldCriterion : function (criterion, fieldName) {
        var result;

        if (criterion.criteria) {
            for (var i = 0; i < criterion.criteria.length; i++) {
                result = this.getFieldCriterion(criterion.criteria[i], fieldName);
                if (result) return result;
            }
        } else {
            if (criterion.fieldName == fieldName) return criterion;
        }

        return null;
    },

    dropOnChange : function (newCriteria, oldCriteria, requestProperties) {
        
        
        if (isc.getKeys(oldCriteria).length != isc.getKeys(newCriteria).length) return -1;
        for (var field in oldCriteria) {
            var oldCriteriaValue = oldCriteria[field],
                newCriteriaValue = newCriteria[field];
            if (isc.isAn.Array(oldCriteriaValue)) {
                
                if (!isc.isAn.Array(newCriteriaValue)) return -1;
                if (oldCriteriaValue.length != newCriteriaValue.length) return -1;
                // the intersection of two identical arrays should be the same length as either of
                // the original arrays
                if (oldCriteriaValue.intersect(newCriteriaValue).length != oldCriteriaValue.length)
                {
                    return -1;
                }
            } else if (isc.isA.Date(oldCriteriaValue) && isc.isA.Date(newCriteriaValue))
            {
                if (oldCriteriaValue.getTime() != newCriteriaValue.getTime()) return -1;
            } else if (oldCriteriaValue != newCriteriaValue) {
                return -1;
            }
        }
        return 0;
    },
    
    
    // dropOnFieldChange() - used for the case where dropOnChange is not true but filter type is
    // exact rather than substring match
    // - if any fields in the old criteria have changed, drop cache
    // - if any new non-dataSource fields have been introduced, drop cache
    // - can retain cache if new criteria restrict previously unrestricted fields (in this case
    //   filter is definitely more restricted)
    dropOnFieldChange : function (newCriteria, oldCriteria, requestProperties) {
        
        var newFields = isc.getKeys(newCriteria),
            oldFields = isc.getKeys(oldCriteria),
            numFieldsDifference = newFields.length - oldFields.length
        ;

        // if new criteria has fewer properties, then it's not more restrictive
        if (numFieldsDifference < 0) return -1;
        
        
        // Ensure that existing criteria are unchanged - otherwise we'll need to drop cache
        for (var field in oldCriteria) {
            var oldCriteriaValue = oldCriteria[field],
                newCriteriaValue = newCriteria[field];
            // If we've dropped a field entirely from the criteria filter is less restrictive
            // (hit server)
            if (newCriteriaValue == null) return -1;
            
            if (isc.isAn.Array(oldCriteriaValue)) {
                
                if (!isc.isAn.Array(newCriteriaValue)) return -1;
                if (oldCriteriaValue.length != newCriteriaValue.length) return -1;
                // the intersection of two identical arrays should be the same length as either of
                // the original arrays
                if (oldCriteriaValue.intersect(newCriteriaValue).length != oldCriteriaValue.length)
                {
                    return -1;
                }
            } else if (isc.isA.Date(oldCriteriaValue) && isc.isA.Date(newCriteriaValue))
            {
                if (oldCriteriaValue.getTime() != newCriteriaValue.getTime()) return -1;
            } else if (oldCriteriaValue != newCriteriaValue) {
                return -1;
            }
        }
        
        // At this point we know the old criteria are unchanged.
        // If the new criteria has more properties, and does not contain any non-DS properties 
        // not present in the old criteria, then it must be more restrictive
        if (numFieldsDifference > 0) {
            newFields.removeList(oldFields);
            // a non-DS field present in newCriteria and not oldCriteria means potentially less
            // restrictive criteria
            for (var i = 0; i < newFields.length; i++) {
                if (this.getField(newFields[i]) == null) return -1;
            }
            return 1;
        }
        return 0;
    },
    
    dropOnShortening : function (newCriteria, oldCriteria, requestProperties) {
        var newFields = isc.getKeys(newCriteria),
            oldFields = isc.getKeys(oldCriteria),
            numFieldsDifference = newFields.length - oldFields.length
        ;

        // if new criteria has fewer properties, then it's not more restrictive
        if (numFieldsDifference < 0) return -1;
        var result = 0;
        for (var field in oldCriteria) {
            var oldCriteriaValue = oldCriteria[field],
                newCriteriaValue = newCriteria[field];
            if (newCriteriaValue == null) return -1;
            // if a field that is not found in the DS changes when we're in dropOnShortening mode,
            // consider that as potentially less restrictive criteria.
			// Also, a field marked ignoreTextmMatchStyle is less restrictive if it changes
            if ((this.getField(field) == null || this.getField(field).ignoreTextMatchStyle) && 
								oldCriteriaValue != newCriteriaValue)
			{
				return -1;
			}
            if (isc.isAn.Array(oldCriteriaValue)) {
                // copied from dropOnChange behavior
                if (!isc.isAn.Array(newCriteriaValue)) return -1;
                if (oldCriteriaValue.length != newCriteriaValue.length) return -1;
                if (oldCriteriaValue.intersect(newCriteriaValue).length != oldCriteriaValue.length)
                {
                    return -1;
                }
            } else if (isc.isA.String(oldCriteriaValue)) {
                if (!isc.isA.String(newCriteriaValue)) return -1;
                if (newCriteriaValue.indexOf(oldCriteriaValue) == -1) return -1;
                if (oldCriteriaValue.length > newCriteriaValue.length) return -1;
                if (oldCriteriaValue.length < newCriteriaValue.length) result = 1;
            } else if (isc.isA.Date(oldCriteriaValue) && isc.isA.Date(newCriteriaValue))
            {
                if (oldCriteriaValue.getTime() != newCriteriaValue.getTime()) return -1;
            } else if (oldCriteriaValue != newCriteriaValue) {
                return -1;
            }
        }
        // at this point, the new criteria is either equivalent or more restrictive, so if the new
        // criteria has more properties, and does not contain any non-DS properties not present
        // in the old criteria, then it must be more restrictive
        if (numFieldsDifference > 0) {
            newFields.removeList(oldFields);
            // a non-DS field present in newCriteria and not oldCriteria means potentially less
            // restrictive criteria
            for (var i = 0; i < newFields.length; i++) {
                if (this.getField(newFields[i]) == null) return -1;
            }
            return 1;
        }
        return result;
    },
    
    //> @method  dataSource.compareDates() (A)
    // Convenience method to compare two Date objects appropriately, depending on whether the
    // passed-in fieldName refers to a field of +link{type:FieldType,type} "datetime" or 
    // "date".  In the former case, the dates are compared using +link{Date.compareDates};
    // in the latter case, or if the supplied fieldName is null or unknown to this DataSource,
    // the dates are compared using +link{Date.compareLogicalDates}.
    // @param date1 (Date) First date in comparison
    // @param date2 (Date) Second date in comparison
    // @param fieldName (String) The name of the field for which the comparison is being run
    // @return (Number) 0 if equal, -1 if first date &gt; second date, 1 if second date &gt;
    //                  first date
    // @visibility external
    //<
    compareDates : function (date1, date2, fieldName, otherFieldName) {
        var field = this.getField(fieldName),
            otherField = otherFieldName ? this.getField(otherFieldName) : null;
        if ((field && field.type == "datetime") || (otherField && otherField.type == "datetime")) {
            return Date.compareDates(date1, date2);
        } else if ((field && field.type == "time") || (otherField && otherField.type == "time")) {
            return isc.Time.compareLogicalTimes(date1, date2);
        } else {
            return Date.compareLogicalDates(date1, date2);
        }
    },
    
    // Convenience method for use by the AdvancedCriteria operator methods - helps to enable 
    // more compact code, because dates need to be compared in a different way to values of
    // other types
    compareValues : function (value1, value2, fieldName, ignoreCase) {
        if (isc.isA.Date(value1) && isc.isA.Date(value2)) {
            return this.compareDates(value1, value2, fieldName);
        } else {
            var v1 = ignoreCase && value1.toLowerCase ? value1.toLowerCase() : value1, 
                v2 = ignoreCase && value2.toLowerCase ? value2.toLowerCase() : value2;
            // Javascript does not consider null to be less than "some string", though it does
            // consider null to be less than "1".  Work around this anomalous behavior.
            if (v1 == null && v2 != null) return 1;
            if (v1 != null && v2 == null) return -1;
            // NOTE: The special return value 2 means that we've been asked to compare two values 
            // that are not equal but also are not sensibly "greater than" or "less than" 
            // one another - for example "Blink" and 182.
            return v1 > v2 ? -1 : (v1 < v2 ? 1 : (v1 == v2 ? 0 : 2))
        }
    },

    //> @method dataSource.recordsAreEqual
    // Convenience method to test if two records are equal. Testing is done only for the
    // fields defined in the DataSource, anything else is ignored.
    //
    // @param record1 (object) record to be compared against.
    // @param record2 (object) record to be compared.
    //
    // @return (boolean) true if the records are equal, false otherwise.
    // @visibility auditing
    //<
    recordsAreEqual: function(record1, record2) {
        var fieldNames = this.getFieldNames();
        for (var i = 0; i <  fieldNames.length; i++) {
            if (this.compareValues(record1[ fieldNames[i] ], record2[ fieldNames[i] ], fieldNames[i]) != 0) {
                return false;
            }
        }
        return true;
    }
});

// AdvancedFilter additions for extended local filtering
// --------------------------------------------------------------------------------------------

//> @object AdvancedCriteria
// AdvancedCriteria is a format for representing search criteria which may include
// operators on field values such as "less than", or may include sub-clauses such as several 
// criteria applied to fields joined by an "OR" operator.
// <P>
// SmartClient DataSources can use AdvancedCriteria to search a list of +link{Record}s, and
// the SmartClient Java Server can translate AdvancedCriteria to either SQL or Hibernate
// queries (<b>Note:</b> The server-side AdvancedCriteria handling feature is only available 
// with the <b>Power</b> and <b>Enterprise</b> Editions of SmartClient; the Pro Edition is 
// limited to ordinary criteria handling on the server side).<p>
// If the entire dataset is cached locally, SmartClient can perform AdvancedCriteria filtering 
// on the client, avoiding a server call.
// <P>
// <var class="smartclient">
// An AdvancedCriteria is an ordinary JavaScript object which can be created directly
// with JavaScript literal notation.  For example:
// <pre>
// var advancedCriteria = {
//        _constructor:"AdvancedCriteria",
//        operator:"and",
//        criteria:[
//            // this is a Criterion
//            { fieldName:"salary", operator:"lessThan", value:"80000" },
//            { operator:"or", criteria:[
//                  { fieldName:"title", operator:"iContains", value:"Manager" },
//                  { fieldName:"reports", operator:"notNull" }
//              ]  
//            }
//        ]
//    }
// </pre>
// An AdvancedCriteria is in effect a +link{Criterion} that has been marked with 
// _constructor:"AdvancedCriteria" to mark it as complete criteria.
// <P>
// This makes AdvancedCriteria very easy to store and retrieve as JSON strings, using
// +link{JSON.encode,JSONEncoder}.
// </var>
// <var class="smartgwt">
// AdvancedCriteria objects can be created directly in java. For example:
// <pre>
// AdvancedCriteria criteria = new AdvancedCriteria(OperatorId.AND, new Criterion[]{
//     new Criterion("salary", OperatorId.LESS_THAN, 8000),
//     new AdvancedCriteria(OperatorId.OR, new Criterion[]{
//         new Criterion("title", OperatorId.ICONTAINS, "Manager"),
//         new Criterion("reports", OperatorId.NOT_NULL)
//     })
// });
// </pre>
// </var>
// <P>
// In addition to building a raw AdvancedCriteria object as described above, the
// +link{DataSource.convertCriteria()} and +link{DataSource.combineCriteria()} methods
// may be used to create and modify criteria based on simple fieldName / value mappings.
// <P>
// When passed to the SmartClient Server, a server-side AdvancedCriteria instance (in the
// package com.isomorphic.criteria) can be retrieved from a DSRequest via
// com.isomorphic.datasource.DSRequest.getAdvancedCriteria().  These same AdvancedCriteria
// objects can be directly created server side, and applied to a DSRequest via
// setAdvancedCriteria().
// <P>
// +link{RestDataSource}, the recommended way of integration with servers that are not running
// the SmartClient Server Framework, defines a standard XML and JSON serialization of
// <code>AdvancedCriteria</code>.
// <P>
// For other servers, you can translate <code>AdvancedCriteria</code> into whatever format is
// expected by the server, typically by implementing +link{dataSource.transformRequest()}.
// <P>
// <var class="smartgwt">
// The internal representation of AdvancedCriteria is a simple JavaScript structure, available
// via AdvancedCriteria.getJsObj():
// <pre>
// // an AdvancedCriteria
// {
//     _constructor:"AdvancedCriteria",
//     operator:"and",
//     criteria:[
//         // this is a Criterion
//         { fieldName:"salary", operator:"lessThan", value:"80000" },
//         { operator:"or", criteria:[
//             { fieldName:"title", operator:"iContains", value:"Manager" },
//             { fieldName:"reports", operator:"notNull" }
//           ]  
//         }
//     ]
// }
// </pre>
// And an AdvancedCriteria can also be created from a JavaScriptObject.  This makes
// AdvancedCriteria very easy to store and retrieve as JSON strings, using
// +link{JSON.encode,JSONEncoder}.
// </var>
// See +link{group:criteriaEditing,Criteria Editing} for information about
// editing AdvancedCriteria in a DynamicForm.
//
// @inheritsFrom Criterion
// @group advancedFilter
// @treeLocation Client Reference/Data Binding/DataSource
// @serverDS allowed
// @visibility external
//<


//> @object Criterion
// An object representing a criterion to apply to a record.  
// <P>
// A criterion is part of the definition of an +link{AdvancedCriteria} object, which is used to
// filter records according to search criteria. 
// <P>
// A criterion consists of an +link{criterion.operator} and typically a
// +link{dataSourceField.name,fieldName} from a
// +link{Record} and a +link{criterion.value,value} to compare to.  However some operators
// either don't require a value (eg, isNull) or act on other criteria rather than directly on a
// +link{Record}'s fields (eg, the "and" and "or" logical operators).
// @group advancedFilter
// @treeLocation Client Reference/Data Binding/DataSource
// @serverDS allowed
// @visibility external
//<

//> @attr criterion.operator (OperatorId : null : IR)
// Operator this criterion applies.
// @group advancedFilter
// @serverDS allowed
// @visibility external
//<

//> @attr criterion.fieldName (String : null : IR)
// Name of the field in each +link{Record} that this criterion applies to.  Not applicable for a
// criterion with +link{criterion.criteria,sub-criteria}. Can be specified as a dataPath to 
// allow matching nested objects. Use '/' as delimiters for dataPath. See 
// +link{type:DataPath,dataPath} for more information.
// @group advancedFilter
// @serverDS allowed
// @visibility external
//<

//> @attr criterion.value (any : null : IR)
// Value to be used in the application of this criterion.
// <P>
// Value may be required or not required, or may be an Array, according to the
// +link{type:OperatorValueType} of the operator.
//
// @group advancedFilter
// @serverDS allowed
// @visibility external
//<

//> @attr criterion.criteria (Array of Criterion : null : IR)
// For a criterion with an operator that acts on other criteria (eg "and", "or"), a list of
// sub-criteria that are grouped together by the operator.
// @group advancedFilter
// @serverDS allowed
// @visibility external
//<

//> @attr criterion.start (any : null : IR)
// Start value of a criterion with an operator of type <code>"valueRange"</code>.
// @group advancedFilter
// @serverDS allowed
// @visibility external
//<


//> @attr criterion.end (any : null : IR)
// End value of a criterion with an operator of type <code>"valueRange"</code>.
// @group advancedFilter
// @serverDS allowed
// @visibility external
//<

// Operators
// ---------------------------------------------------------------------------------------

//> @type OperatorValueType
// Indicates the kind of value expected in a +link{Criterion} that uses this operator.
// 
// @value "fieldType" +link{criterion.value} should contain a value of the same type as the
//                    field.
// @value "fieldName" +link{criterion.value} should be the name of another field in the record
// @value "none"      no criterion.value or other setting required (used for operators like
//                    isNull).
// @value "criteria" +link{criterion.criteria} should be an Array of criteria (used for logical
//                   operators like "and").
// @value "valueRange" +link{criterion.start} and +link{criterion.end} should contain start and
//                     end values, both of the same type as the field.
// @value "valueSet" +link{criterion.value} should contain an Array of valid field values.
// @value "custom" +link{criterion.value} should contain a value which is not necessarily the
//                 same type as the field (used for regexp and similar operators).  
//                 +link{operator.editorType} can be specified as a FormItem to use to
//                 enter a value for the criterion.
// @group advancedFilter
// @treeLocation Client Reference/Data Binding/DataSource
// @serverDS allowed
// @visibility external
//<

//> @type OperatorId
// An operator is used as part of a +link{Criterion} when specifying +link{AdvancedCriteria}.
// <P>
// This list of operators indicates the set of operators built into SmartClient DataSources,
// which can be used for both client and server-side filtering.   You can extend the list of
// operators with +link{dataSource.addSearchOperator()}.
//
// @value "equals" exactly equal to
// @value "notEqual" not equal to 
// @value "iEquals" exactly equal to, if case is disregarded
// @value "iNotEqual" not equal to, if case is disregarded
// @value "greaterThan" Greater than
// @value "lessThan" Less than
// @value "greaterOrEqual" Greater than or equal to
// @value "lessOrEqual" Less than or equal to
// @value "contains" Contains as sub-string (match case)
// @value "startsWith" Starts with (match case)
// @value "endsWith" Ends with (match case)
// @value "iContains" Contains as sub-string (case insensitive)
// @value "iStartsWith" Starts with (case insensitive)
// @value "iEndsWith" Ends with (case insensitive)
// @value "notContains" Does not contain as sub-string (match case)
// @value "notStartsWith" Does not start with (match case)
// @value "notEndsWith" Does not end with (match case)
// @value "iNotContains" Does not contain as sub-string (case insensitive)
// @value "iNotStartsWith" Does not start with (case insensitive)
// @value "iNotEndsWith" Does not end with (case insensitive)
// @value "iBetweenInclusive" shortcut for "greaterOrEqual" + "and" + "lessOrEqual" (case insensitive)
// @value "regexp" Regular expression match
// @value "iregexp" Regular expression match (case insensitive)
// @value "isNull" value is null
// @value "notNull" value is non-null.  Note empty string ("") is non-null
// @value "inSet" value is in a set of values.  Specify criterion.value as an Array
// @value "notInSet" value is not in a set of values.  Specify criterion.value as an Array
// @value "equalsField" matches another field (match case, specify fieldName as criterion.value)
// @value "notEqualField" does not match another field (match case, specify fieldName as criterion.value)
// @value "iEqualsField" matches another field (case insensitive, specify fieldName as criterion.value)
// @value "iNotEqualField" does not match another field (case insensitive, specify fieldName as criterion.value)
// @value "greaterThanField" Greater than another field (specify fieldName as criterion.value)
// @value "lessThanField" Less than another field (specify fieldName as criterion.value)
// @value "greaterOrEqualField" Greater than or equal to another field
//        (specify fieldName as criterion.value)
// @value "lessOrEqualField" Less than or equal to another field
//        (specify fieldName as criterion.value)
// @value "containsField" Contains as sub-string (match case) another field value
//        (specify fieldName as criterion.value)
// @value "startsWithField" Starts with (match case) another field value
//        (specify fieldName as criterion.value)
// @value "endsWithField" Ends with (match case) another field value
//        (specify fieldName as criterion.value)
// @value "iContainsField" Contains as sub-string (case insensitive) another field value
//        (specify fieldName as criterion.value)
// @value "iStartsWithField" Starts with (case insensitive) another field value
//        (specify fieldName as criterion.value)
// @value "iEndsWithField" Ends with (case insensitive) another field value
//        (specify fieldName as criterion.value)
// @value "notContainsField" Does not contain as sub-string (match case) another field value
//        (specify fieldName as criterion.value)
// @value "notStartsWithField" Does not start with (match case) another field value
//        (specify fieldName as criterion.value)
// @value "notEndsWithField" Does not end with (match case) another field value
//        (specify fieldName as criterion.value)
// @value "iNotContainsField" Does not contain as sub-string (case insensitive) another field value
//        (specify fieldName as criterion.value)
// @value "iNotStartsWithField" Does not start with (case insensitive) another field value
//        (specify fieldName as criterion.value)
// @value "iNotEndsWithField" Does not end with (case insensitive) another field value
//        (specify fieldName as criterion.value)
// @value "and" all subcriteria (criterion.criteria) are true
// @value "not" all subcriteria (criterion.criteria) are false
// @value "or" at least one subcriteria (criterion.criteria) is true
// @value "between" shortcut for "greaterThan" + "lessThan" + "and".  Specify criterion.start
//         and criterion.end
// @value "betweenInclusive" shortcut for "greaterOrEqual" + "lessOrEqual" + "and".  Specify
//         criterion.start and criterion.end
// @group advancedFilter
// @treeLocation Client Reference/Data Binding/DataSource
// @serverDS allowed
// @visibility external
//<


//> @object Operator
// Specification of an operator for use in filtering, for example "equals".
// Use with +link{DataSource.addSearchOperator()} to define custom filtering behaviors for
// client-side filtering.
// @group advancedFilter
// @treeLocation Client Reference/Data Binding/DataSource
// @serverDS allowed
// @visibility external
//<
  
//> @attr operator.ID (OperatorId : null : IR)
// Unique id for an operator, which appears within +link{AdvancedCriteria} as the
// +link{operator} property.
// <P>
// A list of built-in identifiers is +link{OperatorId,here}.
// @group advancedFilter
// @serverDS allowed
// @visibility external
//<

//> @attr operator.title (String : null : IR)
// User-visible title for this operator, such as "doesn't contain".
// <P>
// To simplify internationalization by separating titles from operator code, you can use
// specify +link{operator.titleProperty} instead of this property.
// 
// @group advancedFilter
// @serverDS allowed
// @visibility external
//<

//> @attr operator.titleProperty (identifier : null : IR)
// Name of a property on the +link{Operators} class that provides the title for this operator.
// @group advancedFilter
// @serverDS allowed
// @visibility external
//<


//> @method operator.condition()
// Method which actually evaluates whether a given record meets a +link{criterion}.
// <P>
// For operators that act on +link{criterion.criteria,sub-criteria}, call
// +link{dataSource.evaluateCriterion()} to evaluate sub-criteria.
// <P>
// Because criteria are sometimes applied to user-entered data that has not been validated, a
// robust <code>condition()</code> function should expect that data found in a +link{Record}
// may be null, NaN, not the correct type (eg "NA" for a type:"date" field) or otherwise out of
// the expected range.
// <P>
// Note that an Operator has access both to the +link{Criterion} object, allowing operators
// that act on more than one field or perform calculations, and access to the +link{Operator}
// object itself, allowing a <code>condition()</code> function to be shared across a range of
// related operators with different +link{OperatorId}s.
// 
// @param value (any) value from the field supplied as +link{criterion.fieldName}, if
//   applicable
// @param record (Record) record being evaluated
// @param fieldName (any) fieldName supplied as +link{criterion.fieldName}, if applicable
// @param criterion (Criterion) criterion definition
// @param operator (Operator) operator definition
// @return (boolean) whether the field passes this criteria
// @group advancedFilter
// @visibility external
//<

//> @attr operator.fieldTypes (Array of FieldType : null : IR)
// List of types that this Operator is valid for.
// <P>
// If omitted, the operator is assumed to be valid for all FieldTypes unless a list of 
// FieldTypes is passed to +link{DataSource.addSearchOperator}.
// @group advancedFilter
// @serverDS allowed
// @visibility external
//<

//> @attr operator.requiresServer (boolean : false : IR)
// Whether this operator needs to be executed on the server side.
// <P>
// This implies that if a +link{criterion} using this operator is either introduced into
// +link{AdvancedCriteria,criteria} or is changed, the server will need to be contacted to
// perform filtering.
// @group advancedFilter
// @serverDS allowed
// @visibility external
//<

//> @attr operator.hidden (boolean : false : IR)
// Whether this operator should be offered to users by default in interfaces such as the
// +link{class:FilterBuilder}.  
// <P>
// Setting hidden:true means the operator can be used in a programmatic search, for example,
// by calling +link{resultSet.setCriteria()}, but does not appear in the UI.
// @group advancedFilter
// @serverDS allowed
// @visibility external
//<

//> @method operator.compareCriteria()
// Compare two criteria, both of which use this operator, and report whether the newCriteria is
// definitely more restrictive than the previous criteria.
// <P>
// This is used by the +link{ResultSet} to understand whether client-side filtering can
// continue using cached data, or whether server-side filtering must be used instead.
// 
// @param newCriterion (Criterion) new criterion
// @param oldCriterion (Criterion) previous criterion
// @return (Number) 0 if the criteria are equivalent, 1 if newCriterion is guaranteed more
//                      restrictive, and -1 if newCriterion is not guaranteed more restrictive
// @group advancedFilter
// @visibility external
//<

//> @attr operator.valueType (OperatorValueType : null : IR)
// Indicates the kind of value expected in a +link{Criterion} that uses this operator.
// +link{OperatorValueType} lists possibilities.
// <P>
// The default of <code>null</code> is equivalent to "fieldType", indicating that
// +link{criterion.value} is expected to contain a value of the same type as the field
// indicated by +link{criterion.fieldName}.
// @group advancedFilter
// @serverDS allowed
// @visibility external
//<

//> @attr operator.editorType (FormItem classname : null : IR)
// For an operator with +link{valueType}:"custom", indicates what kind of FormItem to use to
// provide a user interface for creating a valid +link{criterion}.  The default of
// <code>null</code> means an ordinary TextItem is fine.
// @group advancedFilter
// @serverDS allowed
// @visibility external
//<

//> @method operator.getCriterion()
// In combination with +link{operator.editorType}, this override point allows you to define a
// client-side only Operator that simply provides a custom UI for creating a Criterion based on
// one of the built-in operators.
// <P>
// For example, the "between" operator allows AdvancedCriteria to be created that can select any
// date range, however in a given application certain specific date ranges might be more
// meaningful (eg "next week", "last quarter") and you might want to offer the user a picker for
// those date ranges.  You could create an operator "presetDateRange" with an editorType 
// indicating a custom SelectItem that shows available ranges, and then implement 
// operation.getCriterion() to take the value from this SelectItem and produce a Criterion 
// selecting the chosen date range.
// <P>
// Note that another approach, if it's not required that this custom interface appear in the
// FilterBuilder, is just to have a separate DynamicForm for picking special date
// ranges, and use +link{DataSource.combineCriteria()} to merge the criteria with the
// FilterBuilder's criteria, as in +explorerExample{dynamicReporting,this sample}.
// 
// @param fieldName (String)
// @param item (FormItem)
// @return Criterion
// @visibility external
//<

//> @attr operator.symbol (String : null : IR)
// The text use when using this operator as an 
// +link{FormItem.allowExpressions, expression} in a FormItem.
// @group advancedFilter
// @serverDS allowed
// @visibility external
//<




// ---------------------------------------------------------------------------------------

//> @attr dataSource.strictSQLFiltering (Boolean : false : IRA)
// If set to true, both client and server-side advanced filtering used by SmartClient will follow
// SQL99 behavior for dealing with NULL values, which is often counter-intuitive to users.
// Specifically, when a field has NULL value, all of the following expressions are false:
// <pre>
//    field == "someValue"  (normally false)
//    field != "someValue"  (normally true)
//    not (field == "someValue")   (normally true)
//    not (field != "someValue")   (normally false)
// </pre>
// This property can be overridden per-query by specifying <code>strictSQLFiltering</code>
// directly as a property on the +link{AdvancedCriteria}.
// <p>
// <b>NOTE:</b> On the server side, this property is only applicable if you are using the 
// SQL DataSource; the other built-in types (Hibernate and JPA/JPA2) do not offer this mode.
// @serverDS allowed
// @visibility external
//< 



isc.DataSource.addClassMethods({

    //> @classMethod DataSource.applyFilter()
    // Returns records in the passed Array that match the provided filter
    // +link{type:Criteria,criteria}. Handles simple or +link{AdvancedCriteria,advanced} criteria.
    // <P>
    // Note that this method differs from the instance method +link{method:dataSource.applyFilter()}
    // in that there will be no field object associated with and of the various criteria passed in.
    //
    // @param   data        (Array)     the list of rows
    // @param   criteria    (Criteria)  the filter criteria
    // @return  (Array)     the list of matching rows
    // @visibility internal
    //<
    // This static implementation is useful for cases where we don't have a DataSource instance
    // but we want to perform simple filtering.
    
    
	applyFilter : function (data, criteria) {
		var output = [];
		if (!data || data.length == 0) return output;
		
		if (this._filterDS == null) {
			this._filterDS = isc.DataSource.create({
				fields:[{name:"_defaultField"}],
				dropUnknownCriteria:false,
				clientOnly:true
			});
		}
		var matches = this._filterDS.applyFilter(data, criteria);
		return matches;
    },

    //> @classMethod DataSource.addSearchOperator()
    // Add a new search operator to all DataSources.  
    // <P>
    // See also +link{method:DataSource.addSearchOperator()} for adding operators to specific
    // DataSources only.
    //
    // @param operator (Operator) definition of the operator to add
    // @group advancedFilter
    // @visibility external
    //<
    addSearchOperator : function (operator) {
        if (!operator || !operator.ID) {
            isc.logWarn("Attempted to add null search operator, or operator with no ID");
            return;
        }
        if (!isc.DataSource._operators) isc.DataSource._operators = {};
        var opList = isc.DataSource._operators,
            undef;
        if (opList[operator.ID] !== undef) {
            // This operator is already registered - log a warning and replace it with the
            // passed-in definition
            isc.logWarn("Attempted to add existing operator " + operator.ID + " - replacing");
            //var index = opList.findIndex("ID", operator.ID);
            //if (index >= 0) opList.removeAt(index);
        }
        isc.DataSource._operators[operator.ID] = operator;
    },

    getSearchOperators : function () {
        return isc.DataSource._operators; 
    },

    //> @classMethod DataSource.setTypeOperators()
    // Set the list of valid +link{OperatorId}s for a given FieldType.
    // 
    // @param typeName (FieldType) 
    // @param operators (Array of OperatorId) available Operators
    // @group advancedFilter
    // @visibility external
    //<
    setTypeOperators : function (typeName, operators) {
        if (!operators ) return;
        if (!isc.isAn.Array(operators) ) operators = [ operators ];
        if (!isc.DataSource._typeOperators ) isc.DataSource._typeOperators = {};
        isc.DataSource._typeOperators[typeName || "_all_"] = operators;
    },
    
    // _getNextRequestId
    // Increment and return a counter for dsRequests
    // This allows each request to have a unique ID within the page
    _currentRequestId:0,
    _getNextRequestId : function () {
        return this._currentRequestId++;
    },
 
    //> @classMethod DataSource.getAutoTitle()
    // Utility method to derive a reasonable user-visible title from an identifier.
    // <P>
    // The following approach is taken:
    // <ul>
    // <li> any underscores (_) or dollar signs ($) become spaces, except that there will never
    //      be either a leading or trailing space.
    // <li> if the fieldName is either entirely uppercase or lowercase, all words separated by
    //      spaces are given a leading capital letter.  Example USER_NAME or user_name -> "User
    //      Name".
    // <li> if there is any use of mixed case, camelCaps convention is assumed, and the field name
    //      is split into separate words based on 1) everywhere an uppercase letter appears after a
    //      lowercase letter 2) everywhere a series of uppercase letters ends.  Letter case will
    //      not be modified, with the exception that the first word will have its first letter 
    //      capitalized.  Example: useHTTPProxy -> "Use HTTP Proxy"
    // </ul>
    //
    // @param identifier (String) identifier for which a title is desired.
    // @return (String) auto-derived title
    //
    // @group title
    // @visibility external
    //<
    getAutoTitle : function (identifier, spaceChars) {
        // allow the regex used for space chars to be replaced.  Sometimes eg "-" makes sense
        spaceChars = spaceChars || new RegExp("[_\$]", "g");

        if (!identifier) return "";
        if (!isc.isA.String(identifier)) identifier = identifier.toString();

        var title;
        // Replace underscores with spaces
        work = identifier.replace(spaceChars, " ");
        // Trim whitespace off the beginning and end - this will also get rid of spare 
        // whitespace introduced by leading or trailing underscores
        var work = work.replace(/^\s+|\s+$/g, "");
        if (work == work.toUpperCase() || work == work.toLowerCase()) {
            // All one case.  Convert to lowercase and then capitalize the first letter
            // of each word
            work = work.toLowerCase();
            var capNext = true;
            title = "";
            for (var i = 0; i < work.length; i++) {
                var letter = work.substr(i, 1);
                if (capNext) {
                    letter = letter.toUpperCase();
                    capNext = false;
                }
                if (letter == ' ') capNext = true;
                title = title + letter;
            }
        } else {
            // camelCase.  A new word starts every place an upper-case letter follows 
            // a lower-case letter, or when a sequence of upper-case letters (eg, HTTP)
            // ends (plus the first letter of the name, of course)
            title = work.substr(0,1).toUpperCase();
            var capLast = work.substr(0,1) == work.substr(0,1).toUpperCase();
            var capSeq = false;
            for (var i = 1; i < work.length; i++) {
                var letter = work.substr(i, 1);
                if (capSeq && letter == letter.toLowerCase()) {
                    capSeq = false;
                    title = title.substr(0, title.length - 1) + " " + 
                            title.substr(title.length - 1);
                }
                if (capLast && letter == letter.toUpperCase()) {
                    capSeq = true;
                }
                if (!capLast && letter == letter.toUpperCase()) {
                    title = title + " ";
                } 
                capLast = letter == letter.toUpperCase();
                title = title + letter;
            }
        }
        return title;
    },

    //> @classMethod dataSource.convertCriteria()
    // Converts criteria expressed in SmartClient's simple criteria format to an AdvancedCriteria
    // object.  
    //
    // @param criteria (Criteria) simple criteria
    // @param [textMatchStyle] (TextMatchStyle) default style of matching text.  Defaults to
    //                                          "substring"
    // @return (AdvancedCriteria) equivalent AdvancedCriteria object
    // @visibility external
    //<
    convertCriteria : function(criteria, textMatchStyle) {
        var aCriteria = {
            _constructor: "AdvancedCriteria",
            operator: "and"
        }
        
        var subCriteria = [];
        for (var fieldName in criteria) {
            if (isc.isA.Array(criteria[fieldName])) {
            
                var disjunct = {
                    _constructor: "AdvancedCriteria",
                    operator: "or",
                    criteria: []
                }
                for (var i = 0; i < criteria[fieldName].length; i++) {
            		var value = criteria[fieldName][i];
                    disjunct.criteria.add({
                        fieldName: fieldName,
                        operator: this.getCriteriaOperator(value, textMatchStyle),
                        value: value
                    });
                }
                subCriteria.add(disjunct);
            } else {
                subCriteria.add({
                    fieldName: fieldName,
                    operator: this.getCriteriaOperator(criteria[fieldName], textMatchStyle),
                    value: criteria[fieldName]
                });
            }
        }

        aCriteria.criteria = subCriteria;
        return aCriteria;
    },
    
    getCriteriaOperator : function (value, textMatchStyle) {
        var operator;
        if (isc.isA.Number(value) || isc.isA.Date(value) || isc.isA.Boolean(value)) {
            operator = "equals";
        } else if (textMatchStyle == "equals" || textMatchStyle == "exact") {
            operator = "iEquals";
        } else if (textMatchStyle == "startsWith") {
            operator = "iStartsWith";
        } else {
            operator = "iContains";
        }
        return operator;    
    },

    
    //> @type CriteriaCombineOperator
    // The logical operator to use when combining criteria objects with the 
    // +link{DataSource.combineCriteria} method.
    // 
    // @value "and"
    // @value "or"
    //
    // @visibility external
    //<

    //> @classMethod dataSource.combineCriteria()
    // Combines two criteria (either simple criteria objects or AdvancedCriteria) using the 
    // "outerOperator".  Note that the combined criteria object will be an AdvancedCriteria
    // unless: <ul>
    // <li>both input criteria objects are simple, and</li>
    // <li>the "outerOperator" is "and", and</li>
    // <li>there is no collision of key names on the two criteria</li>
    // </ul>
    // @param criteria1 (Criteria) first criteria object
    // @param criteria2 (Criteria) second criteria object
    // @param [outerOperator] (CriteriaCombineOperator) operator to use to combine the criteria. 
    //                                          Defaults to "and"
    // @param [textMatchStyle] (TextMatchStyle) style of matching text, if it is necessary to
    //                                          convert a simple criteria object to an 
    //                                          AdvancedCriteria.  Defaults to "substring"
    // @return (Criteria) The combined criteria
    // @example dynamicReporting
    // @visibility external
    //<
    // subCriteria param: Sub-criteria of AdvancedCriteria are not marked with
    // _constructor:"AdvancedCriteria", but in some cases we need to combine them into
    // composite criteria, and we should use the "Advanced" type logic (checking for operator,
    // fieldName etc), and avoid marking the generated object as _constructor:"AdvancedCriteria".
    combineCriteria : function (criteria1, criteria2, outerOperator, textMatchStyle, subCriteria) {
        if (!criteria1) return criteria2;
        if (!criteria2) return criteria1;        
    
        if (!outerOperator) outerOperator = "and";
        
        if (outerOperator != "and" && outerOperator != "or") {
            isc.logWarn("combineCriteria called with invalid outerOperator '" + 
                                outerOperator + "'");
            return null;
        }
    
        var undef, advanced;
        
        // see if both criteria are simple and we can safely combine them as a simple Criteria
        // Note: can't use isAdvancedCriteria because it's an instance method
        if (!subCriteria && criteria1._constructor != "AdvancedCriteria" &&
            criteria2._constructor != "AdvancedCriteria" &&
            outerOperator == "and") 
        {
            for (var key in criteria1) {
                if (criteria2[key] != undef) {
                    advanced = true;
                    break;
                }
            }
        } else {
            advanced = true;
        }
        
        // return simple criteria by straightforward combination
        if (!advanced) {
            return isc.addProperties({}, criteria1, criteria2);
        }
        
        // combine into an AdvancedCriteria
        var advCrit1, advCrit2;
        
        // if either criteria is currently simple, upgrade it to advanced
        if (subCriteria || criteria1._constructor == "AdvancedCriteria") {
            advCrit1 = criteria1;
        } else {
            advCrit1 = isc.DataSource.convertCriteria(criteria1, textMatchStyle);
        }
        
        if (subCriteria || criteria2._constructor == "AdvancedCriteria") {
            advCrit2 = criteria2;
        } else {
            advCrit2 = isc.DataSource.convertCriteria(criteria2, textMatchStyle);
        }
        
        // join the two AdvancedCriteria together with the specified outerOperator
        var aCrit = { operator: outerOperator };
        if (!subCriteria) {
            aCrit._constructor = "AdvancedCriteria";
        }
        
        // Optimization opportunity - if we were passed two criteria with the same
        // operator, and that operator is the same as "outerOperator", we can flatten
        // the structure by removing one intervening level.
        if (advCrit1.operator == outerOperator && advCrit2.operator == outerOperator) {
            aCrit.criteria = [];
            aCrit.criteria.addAll(advCrit1.criteria);
            aCrit.criteria.addAll(advCrit2.criteria);
        } else {
            aCrit.criteria = [advCrit1, advCrit2];
        }
        
        return aCrit;
    },
    
    // When we have advanced criteria objects we can have more layers of nesting than are
    // actually necessary
    // For example:
    // {operator:"and",
    //  criteria:[
    //    {fieldName:"x", value:"y", operator:"z"},
    //    {operator:"and",
    //      criteria:[
    //        {fieldName:"a", value:"b", operator:"c"},
    //        {fieldName:"d", value:"e", operator:"f"}
    //      ]
    //     }
    //   ]
    //  }
    // Could have the inner "and" clause eliminated.
    // This method will simplify advanced criteria by flattening nested and / or clauses
    // as much as possible to make it easier to work with the criteria object
    // returned
    simplifyAdvancedCriteria : function (criteria, returnNull) {
        // If we're passed simple criteria don't attempt to do anything to it!
        if (!this.isAdvancedCriteria(criteria)) {
            return criteria;
        }
        criteria = this._simplifyAdvancedCriteria(criteria, returnNull);
        if (criteria == null) return null;
        criteria._constructor = "AdvancedCriteria";
        return criteria;
    },
    _simplifyAdvancedCriteria : function (criteria, clearEmptyCriteria) {
        var operator = criteria.operator;
        if (operator == "and" || operator == "or") {
            var innerCriteria = criteria.criteria;
            
            // Sanity check - we don't really expect to see empty criteria array as it'd
            // be meaningless
            if (innerCriteria == null || innerCriteria.length == 0) {
            	if (clearEmptyCriteria) return null;
                return criteria;
            }
            
            // Single item in an "and" / "or" block means the outer block is unnecessary
            if (innerCriteria.length == 1) {
                criteria = innerCriteria[0];
                return this._simplifyAdvancedCriteria(criteria, true);
            }
            
            // Nested 'and'/'or' blocks - can combine upwards
            var newInnerCriteria = [];
            for (var i = 0; i < innerCriteria.length; i++) {
                var innerCrit = innerCriteria[i];
                // simplify recursively
                innerCrit = this._simplifyAdvancedCriteria(innerCrit, true);
                if (innerCrit != null) {
					if (innerCrit.operator == operator) {
						newInnerCriteria.addList(innerCrit.criteria);
					} else {
						newInnerCriteria.add(innerCrit);
					}
				}
            }
            
            // Catch the case where we had nested and/or blocks with no sub criteria so end
            // up with an empty object all the way up.
            if (newInnerCriteria.length == 0 && clearEmptyCriteria) return null;
            if (newInnerCriteria.length == 1) return newInnerCriteria[0];
            criteria.criteria = newInnerCriteria;
            return criteria;
        // anything that isn't an 'and' or 'or' can't be simplified further.
        } else {
            return criteria;
        }
    },
    
    //> @classMethod    DataSource.combineFieldData()  ([IA])
    //  Internal method to combine data from a field config object with an underlying DataSource
    // field definition.
    // @param   field   (object)    field to update with DS field defaults.
    // @param dsField (object) dataSource field definition
    // @return  (object)            field with its data combined with the DS data
    // @visibility internal
    //<
    
    combineFieldData : function (localField, dsField) {
        if (dsField == null) return localField;

        for (var propertyName in dsField) {
            // validators should be combined, not overridden
            if (propertyName == "validators" && localField.validators != null && 
                dsField.validators != localField.validators) 
            {
                // If the parent field is using the shared, default validator set and
                // the child field is not of the same base SimpleType as the parent 
                // field, do not inherit the validators.  This prevents us inheriting
                // inappropriate validators in certain common use cases (such as when
                // we have a DS that autoDerives its schema from a SQL table, and then
                // overrides certain fields to be type "boolean" rather than the 
                // discovered type, which will often be "integer" because SQL has no 
                // native boolean type)
                if (dsField.validators._typeValidators) {
                    if (isc.SimpleType.getBaseType(localField.type) != 
                        isc.SimpleType.getBaseType(dsField.type))
                    {
                        continue;
                    }
                }
                
                for (var i =0; i < dsField.validators.length; i++) {
                    var dsValidator = dsField.validators[i];
                    //this.logWarn("comparing validators in field: " + field.name);
                    // This check is required as if 'combineFieldData' gets called more than once
                    // on the same field object (or on 2 field objects pointing to the same 
                    // validators array), we can end up with duplicate validator objects.
                    if (!localField.validators.contains(dsValidator)) {
                        // if the field is using the shared, default validators for the type, 
                        // make a copy before modifying
                        if (localField.validators._typeValidators) {
                            localField.validators = localField.validators.duplicate();
                        }
                        localField.validators.add(dsValidator);
                    }
                }
                continue;
            }

            // datasource properties act as defaults - they don't override
            if (localField[propertyName] != null) continue;
            
            // ignore 'name' field - the component-field may have retrieved this ds field via
            // dataPath but we don't want to write our name onto the component-level field
            // definition
            if (propertyName == "name") continue;

            // copy any properties the field does not have a value for
            localField[propertyName] = dsField[propertyName];
        }
        return localField;
    },
    
    // Record level summary functions
    
    
    //> @type RecordSummaryFunction
    // Function to produce a summary value based on field values within a record.
    // Example usage is record-level summaries in 
    // +link{listGridFieldType,"summary" type ListGrid fields}.
    // <P>
    // SummaryFunctions may be specified in one of 2 ways:<ul>
    // <li>as an explicit function or executable
    // +link{type:stringMethod}. Takes 3 parameters:
    // <br><code>record</code> the record for which the value is being calculated
    // <br><code>fields</code> Array of listGridFields from which summaries should be calculated
    // <br><code>summaryField</code> pointer to the summary type field on which the summary function
    // is being run.</li>
    // <li>As a registered RecordSummaryFunction identifier</li></ul>
    // Note that +link{DataSource.registerRecordSummaryFunction()} may be used to expand the
    // set of registered RecordSummaryFunctions.
    //
    // @value sum Iterates through each field, picking up the numeric field values from the record
    // and summing them.
    // @value avg Iterates through each field, picking up the numeric field values from the record
    // and calculating the mean value.
    // @value max Iterates through each field, picking up the numeric field values from the record
    // and calculating the maximum value.
    // @value min Iterates through each field, picking up the numeric field values from the record
    // and calculating the minimum value.
    // @value multiplier Iterates through each field, picking up the numeric field values from the
    // record and multiplying them together.
    // @visibility external
    //<
    
    // These are documented under type RecordSummaryFunction
    _recordSummaryFunctions:{
        sum : function (record, fields, summaryField) {        
            var sumValue = 0;
            for (var i = 0; i < fields.length; i++) {
                var value = record[fields[i].name],
                    floatValue = parseFloat(value);
                if (isc.isA.Number(floatValue)  && floatValue == value) {
                    sumValue += floatValue;
                } else {
                    // just ignore null / empty values
                    if (value != null && value != isc.emptyString) {
                        // returning null indicates invalid value
                        return null;
                    }
                }
            }
            return sumValue;
        },
        
        avg : function (record, fields, summaryField) {
            var total = 0, count=0;
            for (var i = 0; i < fields.length; i++) {
                var value = record[fields[i].name],
                    floatVal = parseFloat(value);
                if (isc.isA.Number(floatVal) && (floatVal == value)) {
                    count += 1;
                    total += floatVal;
                } else {
                    if (value != null && value != isc.emptyString) {
                        // returning null indicates invalid value
                        return null;
                    }
                }
            }
            return count > 0 ? total/count : null;
        },
        
        max : function (record, fields, summaryField) {
            var max, isNumber;
            for (var i = 0; i < fields.length; i++) {
                var value = record[fields[i].name];
                if (isc.isA.Date(value)) {
                    if (isNumber) return null;
                    if (max == null) max = value.duplicate();
                    else if (max.getTime() < value.getTime()) max = value.duplicate();
                } else {
                    isNumber = true;
                    var floatVal = parseFloat(value);
                    if (isc.isA.Number(floatVal) && (floatVal == value)) {
                        if (max == null) max = floatVal;
                        else if (max < value) max = floatVal;
                    } else {
                        if (value != null && value != isc.emptyString) {
                            // returning null indicates invalid value
                            return null;
                        }
                    }
                }
            }
            return max;
        },
        min : function (record, fields, summaryField) {
            var min, isNumber
            for (var i = 0; i < fields.length; i++) {
                var value = record[fields[i].name];
                if (isc.isA.Date(value)) {
                    if (isNumber) return null;
                    if (min == null) min = value.duplicate();
                    if (value.getTime() < min.getTime()) min = value.duplicate();
                } else {
                    // ignore empty values
                    if (value == null || value == isc.emptyString) continue;

                    isNumber = true;
                    var floatVal = parseFloat(value);
                    if (isc.isA.Number(floatVal) && (floatVal == value)) {
                        if (min == null) min = floatVal;
                        else if (min > value) min = floatVal;
                    } else {
                        // returning null indicates invalid value
                        return null;
                    }
                }
            }
            return min;
        },
        multiplier : function (record, fields, summaryField) {
            var multiplier = 0;
            for (var i = 0; i < fields.length; i++) {
                var value = record[fields[i].name],
                    floatVal = parseFloat(value);                    
                if (isc.isA.Number(floatVal) && (floatVal == value)) {
                    if (i == 0) multiplier = floatVal;
                    else multiplier = (multiplier * floatVal);
                } else {
                    // returning null indicates invalid value
                    return null;
                }
            }
            return multiplier;
        }
        
    },
    
    //> @classMethod DataSource.applyRecordSummaryFunction()
    // Applies a +link{type:RecordSummaryFunction} to a record and returns the result.
    // @param summaryFunction (SummaryFunction) Summary Function or identifier for
    //   registered recordSummaryFunction to execute. If passed in as an explicit function
    //   record, fields and summaryField parameters will be passed through to the function.
    // @param record (DataSourceRecord) Record to retrieve a summary for
    // @param fields (Array of DataSourceFields) Set of fields to include in the summary
    // @param summaryField (DataSourceField) field in which this summary will be displayed.
    // @return (any) summary value for the record
    // @visibility external
    //<
    applyRecordSummaryFunction : function (summaryFunction,record,fields,summaryField) {
        if (!record || !fields) return;
        if (isc.isA.String(summaryFunction)) {
            if (this._recordSummaryFunctions[summaryFunction]) {
                summaryFunction = this._recordSummaryFunctions[summaryFunction]
            } else {
                summaryFunction = isc.Func.expressionToFunction("record,fields,summaryField",
                                                                summaryFunction);
            }
        }
        if (isc.isA.Function(summaryFunction)) return summaryFunction(record,fields,summaryField);
    },
    
    //> @classMethod DataSource.registerRecordSummaryFunction()
    // Register a new standard +link{type:RecordSummaryFunction}. This will then be available
    // by calling +link{DataSource.applySummaryFunction()} and passing in just the new method name.
    // @param methodName (string) identifier for the new summary function
    // @param summaryFunction (function | stringMethod) new summary function implementation.
    //  This method should take 3 parameters: <code>record</code> (the record for which the
    //  summary is being generated), <code>fields</code> (an array of fields to include in the
    //  generated summary) and <code>summaryField</code> (a pointer to the field in which the
    //  summary will be displayed [may be null].
    // @visibility external
    //<
    registerRecordSummaryFunction : function (methodName, summaryFunction) {
        if (isc.isA.String(summaryFunction)) {
            summaryFunction = isc.Func.expressionToFunction("record,fields,summaryField",
                                                            summaryFunction);
        }
        this._recordSummaryFunctions[methodName] = summaryFunction;
    },
    
        
	//> @classMethod dataSource.exportClientData()
    // Exports arbitrary client-side data, with client-side formatters applied, so is suitable 
    // for direct display to users.  This method can be used to export data formatted outside
    // of any kind of visual component.
    // <P>
    // Requires the SmartClient server, but does not rely on any server-side DataSources.  If 
    // you need to intervene in the export process server-side - for example, if you need to
    // do something not directly supported with the exported object, such as attach it to an
    // email - use the +link{method:dataSource.exportClientData,instance method} with an 
    // appropriate +link{class:OperationBinding}, as described in the method documentation.
    // <P>
    // To export unformatted data, see +link{dataSource.exportData, exportData} which does
    // not include client-side formatters, but requires both the SmartClient server and the 
    // presence of server-side DataSources.
    // <P>
    // Note that field +link{DataSourceField.displayFormat,displayFormat} is honored for 
    // "date" and "datetime" fields when exporting direct to Excel; see the displayFormat
    // docs for details.
    //
    // @param data (Array of Record) Array of Records to export
    // @param requestProperties (DSRequest properties) Request properties for the export
    // @param callback (DSCallback) Optional callback.  Note that this is only applicable 
    // if you also specify +link{DSRequest.exportToClient,exportToClient}: false in the 
    // request properties
    // @visibility external
    //<
    
    exportClientData : function (data, requestProperties, callback, ds) {
        var props = requestProperties.exportContext || requestProperties || {},
            format = props && props.exportAs ? props.exportAs : "csv",
            fileName = props && props.exportFilename ? props.exportFilename : "export",
            exportDisplay = props && props.exportDisplay ? props.exportDisplay : "download"
        ;

        var serverProps = {
            showPrompt:false,
            transport: props.exportToClient === false ? "xmlHttpRequest" : "hiddenFrame",
            exportResults: true,
            exportDisplay: exportDisplay,
            downloadResult: !(props.exportToClient === false),
            downloadToNewWindow: (exportDisplay == "window"),
            download_filename: (exportDisplay == "window" ? fileName : null),
            params:props.params
        };
    
        var settings = {
            exportAs: props.exportAs,
            exportDisplay: exportDisplay,
            exportFilename: fileName,
            exportPath: props.exportPath,
            exportToClient: props.exportToClient,
            exportToFilesystem: props.exportToFilesystem,
            exportDelimiter: props.exportDelimiter,
            exportFields: props.exportFields,
            exportHeader: props.exportHeader,
            exportFooter: props.exportFooter,
            exportTitleSeparatorChar: props.exportTitleSeparatorChar,
            lineBreakStyle: props.lineBreakStyle
        };

        if (props.headerSpans) settings.headerSpans = props.headerSpans;
        if (props.exportHeaderSpans) settings.exportHeaderSpans = props.exportHeaderSpans;
        if (props.exportOtherFields) settings.exportOtherFields = props.exportOtherFields;
        if (props.exportShowHeaderSpanTitles != null) {
            settings.exportShowHeaderSpanTitles = props.exportShowHeaderSpanTitles;
        }
        if (props.exportSpanTitleSeparator != null) {
            settings.exportSpanTitleSeparator = props.exportSpanTitleSeparator;
        }

        var opId = requestProperties.operationId;
        if (ds == null || opId == null) {
            isc.DMI.callBuiltin({
                methodName: "downloadClientExport",
                arguments: [ data, format, fileName, exportDisplay, settings ],
                requestParams: serverProps,
                callback: callback
            });
        } else {
            isc.addProperties(serverProps, {parameters: settings});
            ds.performClientExportOperation(opId, data, callback, serverProps);
        }
    },
    
    // Helper method for validation error manipulation:
    
    //> @classMethod DataSource.getSimpleErrors()
    // Getter method for extracting server-side validation errors for an attempted
    // "update" or "add" operation, as a  JS Object where each property name is a 
    // field name from the record and each property value is an array of error message strings.
    // For example:<pre>
    //     {
    //         userId : ["A user with this userId already exists"],
    //         orderId : ["Must be a numeric value", "No Order with ID '6A18294' exists"]
    //     }
    // </pre>
    // The Java API DSResponse.addError(fieldName, errorMessage) is used to send server-side errors
    // to the client.  See the Java Server Reference for details.
    //
    // @param dsResponse (DSResponse) response from which you want to extract the errors
    // @return (Object) Map of fieldName to error messages for the errors included in the DSResponse.
    // @group errorHandling
    // @visibility external
    //<
    getSimpleErrors : function (dsResponse) {
        var complexErrors = dsResponse ? dsResponse.errors : null;
        if (isc.isAn.Array(complexErrors)) {
            if (complexErrors.length > 1) {
                this.logWarn("getSimpleErrors() passed an array of error blocks - " +
                    "dropping all but the first set of errors.");
            }
            complexErrors = complexErrors[0]
        }
        if (complexErrors == null) return null;
        
        // The server hands us errors in the format
        // { recordPath:pathString, fieldName:[{errorMessage:"foo", resultingValue:"moo"}] }
        // Simplify this down to just
        // { fieldName:["error message 1", "error message 2"] }
        
        var simpleErrors = {};
        for (var fieldName in complexErrors) {
            var fieldErrors = complexErrors[fieldName];
            if (fieldName == "recordPath" && !isc.isAn.Object(fieldErrors)) continue;
            
            var errorStrings = [];
            
            if (!isc.isAn.Array(fieldErrors)) fieldErrors = [fieldErrors];
    
            for(var i = 0; i < fieldErrors.length; i++) {
                var error = fieldErrors[i];
                if (isc.isAn.Object(error)) {
                    errorStrings.add(error.errorMessage);
                // handle it already being a raw string
                } else if (error != null) {
                    errorStrings.add(error);
                }
            }
            if (errorStrings.length > 0) {
                simpleErrors[fieldName] = errorStrings;
            }
        }
        return simpleErrors;
    },
    
    recordTimingData : function(object, description, type, millis) {
        if (!isc.Log.logIsInfoEnabled("RpcTabTiming") || !object) return;
        if (millis == null) millis = isc.timeStamp();
        var timing;
        if (object.clientTimingStack) {
            timing = object.clientTimingStack[object.clientTimingStack.length-1];   //Peek
        } else {
            timing = object.clientTiming = {name: "Client processing", start: millis, children: []};
            object.clientTimingStack = [];
            object.clientTimingStack.add(object.clientTiming);  // Push
        }
        if (!timing) {
            isc.logWarn("Unexpected failure to find stack entry logging timing data: " +
                                "description: '" + description + "', type: " + type);
            return;
        }
        if (type == "start") {
            var work = {
                start: millis,
                name: description,
                children: []
            };
            timing.children.add(work);
            object.clientTimingStack.add(work);  //Push
        } else if (type == "end") {
            timing.end = millis;
            object.clientTimingStack.removeAt(object.clientTimingStack.length-1);  //Pop
        } else {
            timing.children.add({
                millis: millis,
                name: description
            });
        }
        
    }

});


isc.DataSource.addMethods({

	//> @method dataSource.supportsAdvancedCriteria()
	// Do fetch and filter operations on this dataSource support being passed 
	// +link{AdvancedCriteria}?
	// <P>
	// For a DataSource to support being passed AdvancedCriteria, it must be 
	// +link{clientOnly,clientOnly:true} or +link{cacheAllData,cacheAllData:true}, or have 
	// server side logic which can process AdvancedCriteria objects passed from the client.
	// <P>
	// AdvancedCriteria are supported on the server for standard 
	// +link{group:sqlDataSource,SQL}, +link{group:hibernateIntegration,Hibernate} and 
    // +link{group:jpaIntegration,JPA} DataSources in SmartClient Enterprise or Power editions
    // (not supported in SmartClient Pro).
	// <P>
	// The framework assumes that custom dataSources support AdvancedCriteria; if you have a 
    // a custom DataSOurce implementation that does not support AdvancedCriteria, you can set
	// the +link{allowAdvancedCriteria} property to false.
	// 
	// @return (Boolean) true if this dataSource supports being passed AdvancedCriteria in fetch
	//  and filter type operations, false otherwise.
	// @visibility external
	//<
	supportsAdvancedCriteria : function () {
		if (this.clientOnly || this.cacheData) return true;
		if (this.allowAdvancedCriteria === false) return false;
		if (this.willHandleAdvancedCriteria === false) return false;
		
		return true;
	},
	
	//> @attr dataSource.allowAdvancedCriteria (boolean : null : IRWA)
	// By default, all DataSources are assumed to be capable of handling 
    // +link{object:AdvancedCriteria} on fetch or filter type operations.  This property may be
    // set to <code>false</code> to indicate that this dataSource does not support 
    // advancedCriteria. See +link{supportsAdvancedCriteria()} for further information on this.
    // <p>
    // <b>NOTE:</b> If you specify this property in a DataSource descriptor 
    // (<code>.ds.xml</code> file), it is enforced on the server.  This means that if you run 
    // a request containing AdvancedCriteria against a DataSource that advertises itself as 
    // <code>allowAdvancedCriteria:false</code>, it will be rejected.
    // @see operationBinding.allowAdvancedCriteria
    // @serverDS allowed
	// @visibility external
	//<
     

    isAdvancedCriteria : function (criteria) {
        return isc.DS.isAdvancedCriteria(criteria, this);
    },

    //> @method dataSource.normalizeAdvancedCriteria()
    // This method returns a normalized form of an AdvancedCriteria (it does not affect the 
    // original object).  Normalization consists of identifying subcriteria that specify a 
    // list of values against an operator that only deals with  single values, and expanding 
    // such a subcriterion into a new "or" subclause.  For example, the criterion:<pre>
    //   { fieldName: "state", operator: "iEquals", value: ["NY", "il", "ca", "Tx"] }
    // </pre>
    // will be expanded into this subclause, which will replace it in the normalized
    // criteria:<pre>
    //   { operator: "or", criteria: [
    //       { fieldName: "state", operator: "iEquals", value: "NY" },
    //       { fieldName: "state", operator: "iEquals", value: "il" },
    //       { fieldName: "state", operator: "iEquals", value: "ca" },
    //       { fieldName: "state", operator: "iEquals", value: "Tx" }
    //     ]
    //   }
    // </pre>
    // For subcriteria that are of a negated type, such as "iNotEqual", we use a new "and" 
    // clause rather than an "or" clause.  This is because if I say:<pre>
    //       { fieldName: "state", operator: "notEqual", value: ["NY", "IL", "CA"] }
    // <pre>
    // I mean that I want all of the states except those three: so logically, not NY <b>and</b>
    // not IL <b>and</b> not CA.
    // <p>    
    // Note that normalization is only carried out for those operators where it makes sense to 
    // do so.  For example, checking if a field <code>equals</code> "A" or "B" or "C", is 
    // clearly a sensible and coherent thing to do.  Equally clearly, checking if a field is 
    // <code>greaterThan</code> 12 or 6 or 9 does not make any sense.  We decide whether to 
    // apply normalization based upon the operator's <code>canNormalize</code> setting.  The 
    // full list of built-in operators which have this flag set is as follows:<li>
    // <ul>"equals", and its variants "iEquals", "notEqual" and "iNotEqual"</ul>
    // <ul>"contains" and its variants</ul>    
    // <ul>"startsWith" and its variants</ul>    
    // <ul>"endsWith" and its variants</ul>    
    // <ul>"equalsField" and "notEqualField"</ul>    
    // <ul>"iEqualsField" and "iNotEqualField"</ul>    
    // <ul>"containsField" and its variants</ul>    
    // <ul>"startsWithField" and its variants</ul>    
    // <ul>"endsWithField" and its variants</ul>    
    // 
    // @param criteria (AdvancedCritiera) The AdvancedCriteria to normalize
    // @return (AdvancedCriteria) The normalized AdvancedCriteria object (the original object
    //                            is not changed)
    // @visibility internal for now
    //<
    normalizeAdvancedCriteria : function (criteria, subCrit) {
        var norm = {};
        if (!subCrit) {
            norm._constructor = "AdvancedCriteria";
            if (criteria.strictSQLFiltering) norm.strictSQLFiltering = true;
        }
        if (criteria.criteria) {
        	// don't break on a single element rather than an array in the criteria slot
        	if (!isc.isAn.Array(criteria.criteria)) {
        		criteria.criteria = [criteria.criteria];
			}
            norm.operator = criteria.operator;
            norm.criteria = [];
            for (var i = 0; i < criteria.criteria.length; i++) {
                norm.criteria.add(this.normalizeAdvancedCriteria(criteria.criteria[i], true));
            }
        } else {
            if (!isc.isAn.Array(criteria.value)) {
                isc.addProperties(norm, criteria);
            } else {
                var operator = this.getSearchOperator(criteria.operator);
                if (!operator || !operator.canNormalize) {
                    isc.addProperties(norm, criteria);
                } else {
                    norm.operator = operator.negate ? "and" : "or";
                    norm.criteria = [];
                    for (var i = 0; i < criteria.value.length; i++) {
                        norm.criteria.add({
                            fieldName: criteria.fieldName,
                            operator: criteria.operator,
                            value: criteria.value[i]
                        });
                    }
                }
            }
        }
        return norm;
    },

    //> @method dataSource.addSearchOperator()
    // Add a new search operator, only to this DataSource.  
    // <P>
    // If an existing +link{Operator} is passed, restricts the set of FieldTypes to which that
    // operator can be applied in this DataSource.
    // <P>
    // See also +link{classMethod:DataSource.addSearchOperator()} for adding operators to all
    // DataSources.
    //
    // @param operator (Operator) definition of the operator to add
    // @param [types] (Array of FieldType) types to which this operator applies
    // @group advancedFilter
    // @visibility external
    //<
    addSearchOperator : function (operator, types) {
        if (!operator || !operator.ID) {
            isc.logWarn("Attempted to add null search operator, or operator with no ID");
            return;
        }

        // Register the "new" operator with the class, regardless of whether it's already 
        // there. If it already exists, it will be replaced, which is the behavior we want.
        isc.DataSource.addSearchOperator(operator);

        // create an object for DS instance-level typeOperators
        if (!this._typeOperators ) this._typeOperators = {};
        // for ops via addSearchOperator(), getTypeOperators() returns them along with the 
        // list of global ones - when ops are added via setTypeOperators(), getTypeOperators() 
        // returns only the specified ops for those types
        if (!this._typeOperators._includeGlobals) this._typeOperators._includeGlobals = {};
        if (types) {
            // First we need to remove it from whatever it's attached to right now
            for (var idx = 0; idx < this._typeOperators.length; idx++) {
                this._typeOperators[idx].remove(operator.ID);
            }
            // And now attach it to the types we've been passed
            for (var idx = 0; idx < types.length; idx++) {
                if (!this._typeOperators[types[idx]]) {
                    this._typeOperators[types[idx]] = [ operator.ID ];
                }
                if (!this._typeOperators[types[idx]].contains(operator.ID)) {
                    this._typeOperators[types[idx]].add(operator.ID);
                }
                this._typeOperators._includeGlobals[types[idx]] = true;
            }
        } else {
            if (!this._typeOperators["_all_"]) {
                this._typeOperators["_all_"] = [ operator.ID];
            }
            if (!this._typeOperators["_all_"].contains(operator.ID)) {
                this._typeOperators["_all_"].add(operator.ID);
            }
        }
    },
    

    //> @method dataSource.getSearchOperator()
    // Get the +link{Operator} definition for an +link{OperatorId}.
    //
    // @param operatorId (OperatorId) the id of the operator
    // @return (Operator) the +link{Operator} definition
    // @group advancedFilter
    // @visibility external
    //<
    getSearchOperator : function (operatorId) {
        return isc.DataSource._operators[operatorId];
    },

    //> @method dataSource.getTypeOperators()
    // Get the list of +link{OperatorId}s available on this DataSource for the given +link{FieldType}.
    // <P>
    // If +link{setTypeOperators()} has been called for this DataSource and FieldType, returns that
    // list, otherwise, returns the set of valid operators for the +link{FieldType} as specified by 
    // +link{simpleType.validOperators}, otherwise, the system-wide set of valid operators for the
    // type as registered via +link{classMethod:DataSource.addSearchOperator()}.
    // 
    // @param typeName (FieldType) 
    // @return (Array of OperatorId) available Operators
    // @group advancedFilter
    // @visibility external
    //<
    getTypeOperators : function (typeName) {
        typeName = typeName || "text";

        var typeOps = [],
            type = isc.SimpleType.getType(typeName),
            dsTypes = this._typeOperators
        ;
        if (dsTypes) {
            
            while (type && !dsTypes[type.name]) {
                if (!type || !type.inheritsFrom) break;
                type = isc.SimpleType.getType(type.inheritsFrom, this);
            }
            if (type) {
                var globals = dsTypes._includeGlobals,
                    includeGlobals = globals && globals[type.name] != false
                ;
                if (dsTypes[type.name]) {
                    // add any type-specific operators
                    typeOps.addList(dsTypes[type.name]);
                }
                // add any operators specified for type "_all_"
                typeOps.addList(dsTypes["_all_"]);
                
                
                if (!includeGlobals && typeOps.length > 0) {
                    if (type.validOperators) {
                        // if the simpleType has validOperators, restrict the list 
                        typeOps = typeOps.intersect(type.validOperators);
                    }
                    return typeOps;
                }
            }
        }

        dsTypes = isc.DataSource._typeOperators;
        // look up the global list of operators for this type
        type = isc.SimpleType.getType(typeName);
        while (type && !dsTypes[type.name]) {
            if (!type || !type.inheritsFrom) break;
            type = isc.SimpleType.getType(type.inheritsFrom, this);
        }
        
        if (type && dsTypes[type.name]) {
            // add type-specific operators
            typeOps.addList(dsTypes[type.name]);
        }
        // add any operators specified for type "_all_"
        typeOps.addList(dsTypes["_all_"]);
        
        if (type && type.validOperators) {
            // if the simpleType has validOperators, restrict the list 
            typeOps = typeOps.intersect(type.validOperators);
        }
        
        typeOps = typeOps.getUniqueItems();
        
        return typeOps;
    },

    //> @method dataSource.setTypeOperators()
    // Set the list of +link{OperatorId}s valid for a given FieldType.
    // 
    // @param typeName (FieldType) 
    // @param operators (Array of OperatorId) available Operators
    // @group advancedFilter
    // @visibility external
    //<
    setTypeOperators : function (typeName, operators) {
        if (!operators) return;
        if (!isc.isAn.Array(operators)) operators = [ operators ];
        if (!this._typeOperators) this._typeOperators = {};
        this._typeOperators[typeName || "_all_"] = operators;
        if (!this._typeOperators._includeGlobals) this._typeOperators._includeGlobals = {};
        this._typeOperators._includeGlobals[typeName] = false;
    },

    //> @method dataSource.getFieldOperators()
    // Get the list of +link{OperatorId}s available for this field.
    // <P>
    // By default, if +link{dataSourceField.validOperators,field.validOperators} is set,
    // returns that list, otherwise returns the result of +link{dataSource.getTypeOperators()}.
    // @param field (String or DataSourceField) Field (or field name) to obtain operators for 
    // @return (Array of OperatorId) available Operators
    // @group advancedFilter
    // @visibility external
    //<
    getFieldOperators : function (field) {
        if (isc.isA.String(field)) field = this.getField(field);
        if (!field) return [];

        if (field && field.validOperators) return field.validOperators;
        var baseFieldType = isc.SimpleType.getType(field.type);
        var type = field.type || "text";
        if (!baseFieldType) type = "text";
        return this.getTypeOperators(type);
    },
    
    //> @method dataSource.getFieldOperatorMap()
    // Get the list of +link{Operator}s available for this field, as a +link{ValueMap} from
    // +link{OperatorId} to the +link{operator.title} specified for the +link{Operator}, or
    // the corresponding property in +link{Operators} if +link{operator.titleProperty} is set.
    // <P>
    // This valueMap is suitable for use in a UI for building queries, similar to the
    // +link{FilterBuilder}, and optionally omits operators marked +link{operator.hidden}:true.
    // <p>
    // It is also possible to have this function return only operators of a given 
    // +link{OperatorValueType}, or everything except operators of that type. This is useful, 
    // for example, if you want to return all the logical operators (like "and"), or everything 
    // except the logical operators.
    //
    // @param field (string or DataSourceField) Field (or field name) to obtain operator map for.
    // @param [includeHidden] (boolean) whether to include Operators marked hidden:true
    // @param [valueType] (OperatorValueType) If passed, returns only operators of this +link{OperatorValueType}
    // @param [omitValueType] (boolean) If set, reverses the meaning of the <code>valueType</code> 
    //        parameter, so operators of that +link{OperatorValueType} are the only ones omitted
    // @return (ValueMap) mapping from +link{OperatorId} to title, as described above
    // @group advancedFilter
    // @see dataSource.getTypeOperatorMap
    // @visibility external
    //<
    getFieldOperatorMap : function (field, includeHidden, valueType, omitValueType) {
        if (isc.isA.String(field)) field = this.getField(field);
        var valueMap = {},
            operators = this.getFieldOperators(field);
        includeHidden = includeHidden || field.validOperators != null;
    
        for (var idx = 0; idx < operators.length; idx++) {
            var operatorId = operators[idx];
            var searchOp = this.getSearchOperator(operatorId);

            if (!searchOp) continue; // no such operatorId

            // searchOp is marked hidden, and we're not including hidden operators
            if (searchOp.hidden && !includeHidden &&
                // and the operator hasn't been explicitly specified on the field (which
                // overrides the default "hidden" property)
                (!isc.isAn.Array(field.validOperators) ||
                 !field.validators.contains(operatorId))) continue;

            // check that the operator is appropriate for the valueType requested (if any)
            if (!valueType || (searchOp.valueType == valueType) == !omitValueType) {
                valueMap[operators[idx]] = searchOp.titleProperty == null ? searchOp.title : 
                    isc.Operators[searchOp.titleProperty];
            }
        }
        return valueMap;
    },
    
    //> @method dataSource.getTypeOperatorMap()
    // Get the list of +link{Operator}s available for this +link{FieldType}, as a +link{ValueMap} from
    // +link{OperatorId} to the +link{operator.title} specified for the +link{Operator}, or
    // the corresponding property in +link{Operators} if +link{operator.titleProperty} is set.
    // <P>
    // This valueMap is suitable for use in a UI for building queries, similar to the
    // +link{FilterBuilder}, and optionally omits operators marked +link{operator.hidden}:true.
    // <p>
    // It is also possible to have this function return only operators of a given 
    // +link{OperatorValueType}, or everything except operators of that type. This is useful, 
    // for example, if you want to return all the logical operators (like "and"), or everything 
    // except the logical operators.
    //
    // @param [type] (FieldType) Type to obtain operator map for. Defaults to "text" if not passed.
    // @param [includeHidden] (boolean) whether to include Operators marked hidden:true
    // @param [valueType] (OperatorValueType) If passed, returns only operators of this +link{OperatorValueType}
    // @param [omitValueType] (boolean) If set, reverses the meaning of the <code>valueType</code> 
    //        parameter, so operators of that +link{OperatorValueType} are the only ones omitted
    // @return (ValueMap) mapping from +link{OperatorId} to title, as described above
    // @group advancedFilter
    // @see dataSource.getFieldOperatorMap
    // @visibility external
    //<
    getTypeOperatorMap : function (type, includeHidden, valueType, omitValueType) {
        var valueMap = {},
            operators = this.getTypeOperators(type);
        for (var idx = 0; idx < operators.length; idx++) {
            var searchOp = this.getSearchOperator(operators[idx]);
            if (searchOp && (!searchOp.hidden || includeHidden)) {
                if (!valueType || (searchOp.valueType == valueType) == !omitValueType)
                valueMap[operators[idx]] = searchOp.titleProperty == null ? searchOp.title : 
                    isc.Operators[searchOp.titleProperty];
            }
        }
        return valueMap;
    },

    //> @method dataSource.recordsAsText()
    // Converts a list of Records to simple text formats with a Record per line and values separated
    // by a configurable separator, including both tab-separated-values and comma-separated-values
    // (aka CSV).
    // <P>
    // In addition to the <code>settings</code> parameter for this method,
    // +link{dataSourceField.exportForceText} can be set.  
    // <P>
    // If two or more different text exports are needed for the same DataSource creating a conflict
    // for any DataSourceField setting, +link{DataSource.inheritsFrom} can be used to create a
    // child DataSource where these settings can be changed without recapitulating all field
    // definitions.
    //
    // @param records (Array of Record) records to convert
    // @param [settings] (TextExportSettings) settings for the export
    // @return (String) records as CSV/TSV (separator can be specified)
    // @visibility external
    //<
    recordsAsText : function (records, settings) {
        if (!settings) settings = isc.TextExportSettings.create();

        var i, field;
        var fieldsToExport = [];

        var fieldNameList = settings.fieldList;
        if (!fieldNameList) fieldNameList = this.getFieldNames();

        for (i = 0; i < fieldNameList.length; i++) {
            var name = fieldNameList[i];
            field = this.fields[name];
            if (field) fieldsToExport.add(field);
            else fieldsToExport.add(name);
        }
        var result = "";
        for (i = 0; i < records.length; i++) {
            if (i > 0) result += settings.lineSeparator;
            result += this._recordAsText(records[i], fieldsToExport, settings);
        }
        return result;
    },

    _recordAsText : function (record, fields, settings) {
        var value, valueType;

        var forceText = settings.forceText;
        var escapeChar = settings.getEscapingModeEscapeChar();

        for (var i = 0, result = ""; i < fields.length; i++, result += value) {
            if (i > 0) result += settings.fieldSeparator;

            var field = fields[i];
            if (isc.isA.String(field)) {
                value = record[field];
                valueType = "text";
            } else {
                valueType = field.type;
                value = record[field.name];
                if (field.exportForceText) {
                    forceText = field.exportForceText;                    
                }
            }
            var mappedToString = false;
            if (settings.useDisplayValue && field.name != null) {
                value = this.getDisplayValue(field.name, value);
                if (value != null) { 
                    value = value.toString(); mappedToString = true;
                }
            }
            if (!mappedToString) {
                value = isc.DataSource._getTypedValueAsString(value, valueType, settings);
            }
            if (valueType == "text") switch (forceText) {
            case "leadingSpace":
                value = " " + value;
                break;
            case "formula":
                value = "=" + value;
            }
            value = value.replace(/(\")/g, escapeChar + "$1");
            if (settings.quoteValues) value = "\"" + value + "\"";
        }
        return result;
    },

    //> @method dataSource.recordsFromText()
    // Derive a list of Records from Microsoft Excel-compatible tab-separated-values format, using
    // the current DataSource field order, or an explicitly specified list of fields.
    // <P>
    // If a specified field does not exist in the DataSource, it's assumed the values for that
    // field should end up as Strings.
    // 
    // @param text (String) records as CSV/TSV (separator can be specified)
    // @param [settings] (TextImportSettings) optional settings for the import
    // @return (Array of Record) records derived from TSV
    // @visibility external
    //<
    recordsFromText : function (text, settings) {

        if (!text) return [];

        if (!settings) settings = isc.TextImportSettings.create();

        text = settings.addFinalLineSeparatorIfNotPresent(text);

        var fieldNameList = settings.fieldList;
        if (!fieldNameList) fieldNameList = this.getFieldNames();

        if (settings.hasHeaderLine) {
            fieldNameList = [];
            text = this._processAndRemoveHeaderLine(text, settings, fieldNameList);
        }
        if (fieldNameList.length == 0) return [];

        var dataSource = this, records = [];

        var record = {}, nFields = 0;

        var fieldFunction = function (value) {
            if (nFields < fieldNameList.length) {
                record[fieldNameList[nFields++]] = value;
            }
        };
        var lineFunction = function () {
            if (!isc.TextImportSettings._importAsJsonObjects) {
                isc.Validator._acceptExcelFormats = true;
                dataSource.validateJSONRecord(record, true);
                isc.Validator._acceptExcelFormats = false;
            }
            records.add(record);
            record = {};
            nFields = 0;
            return false;            
        };
        settings.parseTextAndApplyFunctions(text, fieldFunction, lineFunction);
        return records;
    },

    _processAndRemoveHeaderLine : function (text, settings, fieldList) {

        var titleMap = {}, nameMap = {};
        if (!isc.TextImportSettings._importAsJsonObjects) {
            var fieldNames = this.getFieldNames();
            for (var i = 0; i < fieldNames.length; i++) {
                var field = this.getField(fieldNames[i]);
                if (field.name  != null)  nameMap[field.name.toUpperCase()]  = field;
                if (field.title != null) titleMap[field.title.toUpperCase()] = field;
            }
        }
        var fieldFunction = function (value) {
            var target = value.toUpperCase().replace(/^(\s)*/, "").replace(/(\s)*$/, "");
            if     (titleMap[target]) fieldList.add(titleMap[target].name);
            else if (nameMap[target]) fieldList.add(nameMap [target].name);
            else                      fieldList.add(value);
        };
        var lineFunction = function () {
            return true;
        };
        return settings.parseTextAndApplyFunctions(text, fieldFunction, lineFunction);
    }

});

//> @attr dataSourceField.validOperators (Array of OperatorId : null : IR)
// List of operators valid on this field.  
// <P>
// If not specified, all operators that are valid for the field type are allowed.
// @group advancedFilter
// @serverDS allowed
// @visibility external
//<

//> @attr simpleType.validOperators (Array of OperatorId : null : IR)
// Set of search operators valid for this type.  
// <P>
// If not specified, the +link{inheritsFrom,inherited} type's operators will be used, finally
// defaulting to the default operators for the basic types (eg, integer).
// @group advancedFilter
// @visibility external
//<




// Filtering
// ---------------------------------------------------------------------------------------

isc.DataSource.addMethods({ 
  
    //> @method dataSource.evaluateCriterion()
    // Evaluate the given criterion with respect to the passed record.
    // <P>
    // Typically called by the +link{operator.condition,condition} function of a custom
    // +link{Operator} to evaluate +link{criterion.criteria,sub-criteria}.
    //
    // @param record (Record) record to evaluate
    // @param criterion (Criterion) criterion to use
    // @return (boolean) whether the record meets the supplied +link{Criterion}
    // @group advancedFilter
    // @visibility external
    //<
    evaluateCriterion : function (record, criterion) {

        
        if (criterion.requiresServer == true) return true;
        
        var op = this.getSearchOperator(criterion.operator);
        if (op == null) {
            isc.logWarn("Attempted to use unknown operator " + criterion.operator);
            return false;
        }
        
        var checkValidOps = true;
        if (this._applyingCriteriaToList) {
        	for (var i = 0; i < this._testedCriteria.length; i++) {
        		if (this._testedCriteria[i] == criterion) {
        			// we've already tested this criterion (as part of this loop), so no need to test
        			// again
        			checkValidOps = false;
        			// If we determined we should just 'drop' the criteria -- IE it points to
        			// an unknown field and dropUnknownCriteria is true, just return true.
        			if (this._shouldDropCriteria[i] == true) {
        				return true;
					}
				}
			}
		}
        if (checkValidOps) {
        	var testedCriteriaIndex;
			// If we're looping through a list of records applying criteria, record the
			// criteria we've checked for validity so we don't have to re-test for each record
			// in the grid.
			if (this._applyingCriteriaToList) {
				testedCriteriaIndex = this._testedCriteria.length;
				this._testedCriteria[testedCriteriaIndex] = criterion;
			}

            if (criterion.fieldName) {
                var ds = this,
                    field = this.getField(criterion.fieldName);
                if (field == null) {
                    ds = this.getDataSourceForDataPath(criterion.fieldName);
                    field = this.getFieldForDataPath(criterion.fieldName);
                    if (ds == null) {
                        ds = this;
                    }
                }
                
                if (field == null) {
                    if (this.dropUnknownCriteria) {
                        this.logInfo("evaluateCriterion passed criterion for field not explicitly " +
                            "listed in this dataSource:" + criterion.fieldName + 
                            " - dropping this filter","AdvancedCriteria");
                        
                        // If applying criteria to a list, record that we should just ignore this
                        // criterion and return true regardless of the record values.
                        if (this._applyingCriteriaToList) {
                        	this._shouldDropCriteria[testedCriteriaIndex] = true;
                        }
                        return true;
                    } else {
                        this.logInfo("evaluateCriterion passed criterion for field not explicitly " +
                            "listed in this dataSource:" + criterion.fieldName + 
                            " - continuing with filter", "AdvancedCriteria");
                    }
                    
                } else {
                    var validOps = ds.getFieldOperators(field.name);
                    if (!validOps.contains(op.ID)) {
                        this.logWarn("Operator " + op.ID + " is not valid for field " + criterion.fieldName +
                                    ". Continuing anyway.");
                    }
                }
            }
        }

        var field = this.getField(criterion.fieldName);
        var fieldValue = isc.DataSource.getPathValue(record, criterion.fieldName, field);
        var isDateField = field && (isc.SimpleType.inheritsFrom(field.type, "date") || isc.SimpleType.inheritsFrom(field.type, "datetime"));

        var convertToAbsoluteDateOrKeepOriginalValue = function(value) {
            if (isDateField && !isc.isA.Date(value) && isc.DateUtil.isRelativeDate(value)) {
                return isc.DateUtil.getAbsoluteDate(value);
            }

            return value;
        }

        var criterionValues = {
            value: convertToAbsoluteDateOrKeepOriginalValue(criterion.value),
            start: convertToAbsoluteDateOrKeepOriginalValue(criterion.start),
            end: convertToAbsoluteDateOrKeepOriginalValue(criterion.end)
        };

        // If this is a field comparison operator, lets add the value of the other
        // field to the criterionValues object.
        if (op.valueType === "fieldName") {
            var otherField = this.getField(criterionValues.value);
            var otherFieldValue = isc.DataSource.getPathValue(record, criterionValues.value, otherField);
            criterionValues.otherValue = convertToAbsoluteDateOrKeepOriginalValue(otherFieldValue);
        } else if (op.valueType === "criteria") {
            // If we're dealing with a criteria operator such as AND, NOT & OR, lets add the criterion
            // to the criterionValues object.
            criterionValues.criterion = criterion;
            criterionValues.record = record;
        }

        var condition = false;

        // If this is a multiple: true field, we will need to run the condition check several times
        // depending on field type.
        if (field && field.multiple) {
            if (isc.isAn.Array(fieldValue) && !isc.isAn.Array(criterionValues.otherValue)) {
                var fieldDataSource = isc.DataSource.get(field.type);

                for(var i = 0; i < fieldValue.length; i++) {
                    var fieldValueToTest = fieldValue[i];

                    // If the fieldValueToTest is an object, this probably means that the field is
                    // of DataSource type and then we need to use the primaryKey fields value.
                    if (isc.isA.Object(fieldValueToTest) && fieldDataSource) {
                        if (fieldValueToTest[fieldDataSource.getPrimaryKeyFieldName()]) {
                            fieldValueToTest = fieldValueToTest[fieldDataSource.getPrimaryKeyFieldName()];
                        } else {
                            isc.logWarn("The field '" + criterion.fieldName + "' being tested is a multiple value" +
                                "field with a DataSource type but either the DataSource does not exist" +
                                "or it does not have a primary key field. Skipping criterion.");
                            break;
                        }
                    }

                    condition = op.condition(criterion.fieldName, fieldValueToTest, criterionValues, this, isDateField);

                    if (condition) {
                        break;
                    }
                }
            } else {
                if (isc.isAn.Array(fieldValue) && isc.isAn.Array(criterionValues.otherValue) &&
                    op.ID !== "equalsField" && op.ID !== "iEqualsField" &&
                    op.ID !== "notEqualField" && op.ID !== "iNotEqualField") {

                    // If we're comparing fields and both values are arrays and we're not using the correct
                    // operator type, lets log a warning.
                    isc.logWarn("Only 'equalsField', 'iEqualsField', 'notEqualField' and 'iNotEqualField' " +
                        "are valid operators when comparing two multiple: true fields. Ignoring criterion with operator '" + criterion.operator + "'.");
                    condition = true;
                } else {
                    condition = op.condition(criterion.fieldName, fieldValue, criterionValues, this, isDateField);
                }
            }
        } else {
            condition = op.condition(criterion.fieldName, fieldValue, criterionValues, this, isDateField);
        }

        // If the operator is negated, lets return a negated condition
        return (op.negate ? !condition : condition);
    },
    
    recordsMatchingAdvancedFilter : function (data, criteria, requestProperties) {
        var matches = [];
        
        // This flag is needed to match SQL behavior that treats null values as completely unable to
        // be compared, such that (condition-involving-null) == !(condition-involving-null) == false
        this._withinLogicalNot = false;
        
        // This flag dictates whether we match the aforementioned SQL behavior with null values,
        // or use our own, more sophisticated (and logical) algorithms
        this._strictMode = criteria.strictSQLFiltering;
        if (this._strictMode == null) this._strictMode = this.strictSQLFiltering;

        
    	var recursiveCall = false;
        if (!this._applyingCriteriaToList) {
	        this._startApplyingCriteriaToList();
		} else {
			
			recursiveCall = true;
		}

        for (var idx = 0; idx < data.length; idx++) {
        
            // The AdvancedCriteria system makes this very easy - just call evaluateCriterion
            // on the top-level criterion, and it handles all the recursion and evaluation of
            // sub-criteria that it needs to do automatically.
            if (!criteria.operator || this.evaluateCriterion(data[idx], criteria)) {
                matches.add(data[idx]);
            }
        }
        if (!recursiveCall) this._endApplyingCriteriaToList();
        
        return matches;
    },
    
    
    _startApplyingCriteriaToList : function () {
    	this._applyingCriteriaToList = true;
    	this._testedCriteria = [];
    	this._shouldDropCriteria = [];
    },
    _endApplyingCriteriaToList : function () {
    	delete this._testedCriteria;
    	delete this._shouldDropCriteria;
    	delete this._applyingCriteriaToList;
    },
    
    compareAdvancedCriteria : function (newCriterion, oldCriterion, requestProperties) {
    	// run through simplifier so we can compare nested and/or's etc correctly.
    	// Pass in the parameter to return null if the criterion turns out to be
    	// an and/or with no sub criteria so we can identify this case (basically empty criteria).
    	newCriterion = isc.DataSource.simplifyAdvancedCriteria(newCriterion, true);
    	oldCriterion = isc.DataSource.simplifyAdvancedCriteria(oldCriterion, true);
    	if (oldCriterion == null) return newCriterion == null ? 0 : 1;
    	if (newCriterion == null) return -1;

        // Normalize the criteria so that the comparison of arrays will work correctly
        newCriterion = this.normalizeAdvancedCriteria(newCriterion);
        oldCriterion = this.normalizeAdvancedCriteria(oldCriterion);

        var op = this.getSearchOperator(oldCriterion.operator);
        var newOp = this.getSearchOperator(newCriterion.operator);

        // If the operation and the new operation are both undefined return zero to indicate they
        // are the same and do not bother with the rest of the logic.
        if (!op && !newOp) {
            return 0;
        }

        if (op != newOp) {
        	// Usually 2 different operators essentially means there's no way to compare
        	// the 2 criteria and determine which is more restrictive.
        	// In this case we return -1 (code for new criteria being less restrictive) indicating
        	// that a new server-fetch is required.
        	//
        	// AND / OR are special cases - they could contain a subcriterion with an operator 
        	// matching this one - for example:
        	//   {operator:"equals", value:"foo"} 
        	// and
        	//   {operator:"or", 
        	//		criteria:[{operator:"equals", value:"foo"}, {operator:"equals", value:"moo"}]}
        	//
        	// This leads to a couple of cases where we can perform a meaningful comparison and
        	// potentially return +1 [more restrictive] with mismatch operators. Specifically:
        	// - old criteria is "OR" and contains subcriterion that matches, or is known 
        	//   less restrictive than new criteria -- additional subcriteria of the OR make 
        	//   it less restrictive still, so we know the new criteria is more restrictive and
        	//   should return +1
        	// - new criteria is "AND" and contains subcriterion that matches or is known more
        	//   restrictive than old criterion. In this case additional subcriteria on the new crit
        	//   make it more restrictive still, so return +1.
        	
        	
            var rtnVal = -1;
        	if (op && op.ID == "or") {
        		var extraLayer = {operator:"or", criteria:[newCriterion]};
        		rtnVal = op.compareCriteria(extraLayer, oldCriterion, op, this);
        	}
            if (rtnVal != -1) return rtnVal;
            if (newOp && newOp.ID == "and") {
        		var extraLayer = {operator:"and", criteria:[oldCriterion]};
        		return newOp.compareCriteria(newCriterion, extraLayer, newOp, this);
        	}
        	
            
            return -1;
        }        

        return op.compareCriteria(newCriterion, oldCriterion, op, this);

    },
    
    //> @method dataSource.splitCriteria
    // Split a criteria apart based on <code>fields</code>. A new simple criteria
    // is returned with any criteria applicable to the specified fields. The passed
    // <code>criteria</code> is then <u>modified</u> to remove these fields resulting in
    // two distinct criteria.
    // <P>
    // Incoming criteria can be a simple or advanced criteria. For an +link{AdvancedCriteria}
    // only a single level of criteria with a top-level operator of "and" is supported.
    // <P>
    // To avoid modifying an original criteria, use +link{dataSource.copyCriteria} to
    // make a copy to be passed in.
    //
    // @param criteria (Criteria) criteria to be split. May be modified if criteria is
    //                            extracted.
    // @param fields (Array of String) list of fields to extract from criteria
    // @return (Criteria) extracted criteria
    // @visibility external
    //<
    splitCriteria : function (criteria, fields) {
        var splitCriteria = {},
            sourceCriteria = criteria,
            advanced = false
        ;
        if (criteria == null || fields == null) return splitCriteria;

        // Convert advancedCriteria to a simple one
        if (this.isAdvancedCriteria(criteria)) {
            sourceCriteria = this._convertAdvancedCriteriaToSimple(criteria);
            advanced = true;
        }
        if (sourceCriteria == null) return splitCriteria;

        // Simple criteria is just an object with properties that match field names
        // and values that are to be matched.
        for (var i = 0; i < fields.length; i++) {
            var field = fields[i];
            if (sourceCriteria[field] != null) {
                splitCriteria[field] = sourceCriteria[field];

                // Remove field from original criteria
                if (!advanced) {
                    // Simple criteria
                    delete criteria[field];
                } else {
                    if (criteria.fieldName) {
                        // Advanced criteria, stand-alone criteria
                        delete criteria.fieldName;
                        delete criteria.value;
                    } else {
                        // Advanced criteria, multiple criteria
                        for (var c = 0; c < criteria.criteria.length; c++) {
                            var crit = criteria.criteria[c];
                            if (crit.fieldName == field) {
                                criteria.criteria.removeAt(c);
                                break;
                            }
                        }
                    }
                }
            }
        }

        // Clean up original advancedCriteria if it no longer includes any criteria
        if (advanced && criteria.criteria && criteria.criteria.length == 0) {
            // Effectively return client-side criteria to a empty simple criteria
            delete criteria.criteria;
            delete criteria.operator;
            delete criteria._constructor;
        }
        return splitCriteria;
    },

    // splitCriteria helper
    _convertAdvancedCriteriaToSimple : function (criteria) {
        var simpleCriteria = {};

        // Is this a standalone criteria (i.e. no operator)
        if (criteria.fieldName) {
            if (criteria.operator != "equals") {
                isc.logWarn("splitCriteria: advanced criteria operator, " +
                            criteria.operator + ", not supported.");
                return null;
            }
            simpleCriteria[criteria.fieldName] = criteria.value;
            return simpleCriteria;
        }

        // Only support a high-level operator of "And"
        if (criteria.operator != "and") {
            isc.logWarn("splitCriteria: advanced criteria top-level operator, " +
                        criteria.operator + ", not supported.");
            return null;
        }

        // Check out each criterion. Build simple criteria object.
        for (var i = 0; i < criteria.criteria.length; i++) {
            var crit = criteria.criteria[i];
            if (crit.operator != "equals") {
                isc.logWarn("splitCriteria: advanced criteria second-level operator, " +
                            criteria.operator + ", not supported.");
                return null;
            }
            simpleCriteria[crit.fieldName] = crit.value;
        }

        return (isc.isAn.emptyObject(simpleCriteria) ? null : simpleCriteria);
    }

});


// -------------------------------------------------------------------------------------------------
// DataSource.stringMethods
isc.DataSource.registerStringMethods({
    // register transformResponse and transformRequest as stringMethods - this
    // allows observation of parameters
    transformRequest:"dsRequest",
    transformResponse:"dsResponse,dsRequest,data"
    
});



// Define temporary function to init the built-in operators - called at the end of the definition
isc._initBuiltInOperators = function () {

    
    var equality = function (fieldName, fieldValue, criterionValues, dataSource, isDateField) {
        
        if (dataSource._strictMode) {
            if (fieldValue == null || criterionValues.value == null) {
                return (this.negate ? !dataSource._withinLogicalNot : dataSource._withinLogicalNot);
            }
        }

        if (isc.isA.Date(criterionValues.value) && isc.isA.Date(fieldValue)) {
            return (dataSource.compareDates(criterionValues.value, fieldValue, fieldName) == 0);
        } else {
            return (criterionValues.value == fieldValue);
        }
    };

    var rangeCheck = function (fieldName, fieldValue, criterionValues, dataSource, isDateField, nullMeansNull) {
        var start = criterionValues.start || criterionValues.value;
        var end = criterionValues.end || criterionValues.value;

        if (dataSource._strictMode) {
            if (fieldValue == null || (this.lowerBounds && start == null) ||
                (this.upperBounds && end == null)) {
                return  dataSource._withinLogicalNot;
            }
        }

        

        var lowerBoundsCheck = true;
        var upperBoundsCheck = true;
        var isDate = isc.isA.Date(fieldValue);
        var isNumber = isc.isA.Number(fieldValue);
        var isString = isc.isA.String(fieldValue);

        if (this.lowerBounds && start && ((isNumber && isNaN(start)) ||
            (start && isDate && (!isc.isA.Date(start))) ||
            (start && isString && (!isc.isA.String(start) && !isc.isA.Number(start))))) {
            return false;
        }

        if (this.upperBounds && end &&
            ((isNumber && isNaN(end)) ||
                (end && isDate && (!isc.isA.Date(end))) ||
                (end && isString && (!isc.isA.String(end) && !isc.isA.Number(end))))) {
            return false;
        }

        var undef;
        if (!nullMeansNull) {
            if (start === null || start === undef) {
                lowerBoundsCheck = false;
            }

            if (end === null || end === undef) {
                upperBoundsCheck = false;
            }

            if (isDate && !isc.isA.Date(start)) lowerBoundsCheck = false;
            if (isDate && !isc.isA.Date(end)) upperBoundsCheck = false;
        }

        isDate = isc.isA.Date(start) || isc.isA.Date(end);
        isNumber = isc.isA.Number(start) || isc.isA.Number(end);
        isString = isc.isA.String(start) || isc.isA.String(end);
        var ignoreCase = isString && this.caseInsensitive;

        if (fieldValue === null || fieldValue === undef) {
            if (isDate) {
                fieldValue = new Date(-8640000000000000);
            } else if (isNumber) {
                fieldValue = Number.MIN_VALUE;
            } else {
                fieldValue = "";
            }
        } else {
            if (isNumber && isNaN(fieldValue)) {
                start = "" + start;
                end = "" + end;
            }
            // So, we've been handed a Date object to test against a text data value. Whatever we might
            // try to do here will seem wrong to somebody. So, in the absence of anything more sensible,
            // we'll just return false (so all records are excluded when you have this kind of mismatch)
            if (isDate && !isc.isA.Date(fieldValue)) {
                return false;
            }
        }

        // >, >=, lower bounds check on between, betweenInclusive
        if (this.lowerBounds && lowerBoundsCheck) {
            if (dataSource.compareValues(start, fieldValue, fieldName, ignoreCase) <= (this.inclusive ? -1 : 0)) {
                return false;
            }
        }

        // <, <=, upper bounds check on between, betweenInclusive
        if (this.upperBounds && upperBoundsCheck) {
            if (dataSource.compareValues(end, fieldValue, fieldName, ignoreCase) >= (this.inclusive ? 1 : 0)) {
                return false;
            }
        }

        return true;
    };

    var stringComparison = function (fieldName, fieldValue, criterionValues, dataSource, isDateField) {
        if (isc.isA.Number(fieldValue)) {
            fieldValue = "" + fieldValue;
        }

        if (isc.isA.Boolean(fieldValue)) {
            isc.logWarn("Attempt to compare a Boolean as String, testing value " + criterionValues.value +
                " with operator " + this.ID +
                ". This is an invalid comparison, so we are returning FALSE.");
            return false;
        }

        if (isc.isA.Date(fieldValue)) {
            isc.logWarn("Attempt to compare a Date as String, testing value " + criterionValues.value +
                " with operator " + this.ID +
                ". This is an invalid comparison, so we are returning FALSE.");
            return false;
        }

        // Special-case code to match server-side exception when we get a request to do a string-
        // match on a non-text field (note that numbers are OK - they are converted above)
        if (fieldValue != null && !isc.isA.String(fieldValue)) {
            return true;
        }

        // - a null data value cannot contain anything, including null.
        // - a non-null data value is considered to contain null. We originally followed Javascript
        //   and considered non-null values to NOT contain null; however, it is not possible to implement
        //   this scheme in Hibernate without resorting to hackery.  It was easier to change the
        //   client-side rule
        if (fieldValue == null) {
            if (dataSource._strictMode) {
                return (this.negate ? !dataSource._withinLogicalNot : dataSource._withinLogicalNot);
            } else {
                return false;
            }
        }

        var test = criterionValues.value || "";

        if (isc.isA.Number(test)) {
            test = "" + test;
        }

        if (!isc.isA.String(test) || !isc.isA.String(fieldValue)) {
            return false;
        }

        
        if (this.caseInsensitive) {
            fieldValue = fieldValue.toLowerCase();
            test = test.toLowerCase();
        }

        var result = false;

        if (this.startsWith) {
            result = isc.startsWith(fieldValue, test);
        } else if (this.endsWith) {
            result = isc.endsWith(fieldValue, test);
        } else if (this.equals) {
            result = (fieldValue == test);
        } else {
            result = isc.contains(fieldValue, test);
        }

        return result;
    };

    var nullCheck = function(fieldName, fieldValue, criterionValues, dataSource, isDateField) {
        
        return (fieldValue == null);
    };

    var regexpCheck = function(fieldName, fieldValue, criterionValues, dataSource, isDateField) {
        // Bail out early if value is undefined.  This is to provide cross-browser consistency,
        // because Moz always returns false for a test of RegExp(undefined), whereas all the
        // others (IE, Safari and Chrome tested) return true.  They all return false if the
        // value is null, so we'll mirror that and return false for undefined as well
        if (!criterionValues.value) {
            return false;
        }

        // Dates make no sense for regexp (numbers work OK)
        if (isc.isA.Date(criterionValues.value) || isc.isA.Date(fieldValue) || isc.DateUtil.isRelativeDate(criterionValues.value)) {
            return false;
        }

        var regex;

        if (this.caseInsensitive) {
            regex = new RegExp(criterionValues.value, "i");
        } else {
            regex = new RegExp(criterionValues.value);
        }

        return regex.test(fieldValue);
    };

    var matchesPatternCheck = function(fieldName, fieldValue, criterionValues, dataSource, isDateField) {
        // If for some reason the user has set the patternEscapeChar to an empty or undefined
        // value, make sure we abort here.
        if (!dataSource.patternEscapeChar || dataSource.patternEscapeChar.length > 1) {
            throw "dataSource.patternEscapeChar cannot be null or undefined";
        }

        // Lets prep for the state machine. We need the wildcards in arrays to make it easier
        // and more readable below where we check if the character being looked at is a wildcard.
        var singleWildcards = isc.isA.Array(dataSource.patternSingleWildcard) ? dataSource.patternSingleWildcard : [dataSource.patternSingleWildcard || "?", "%"];
        var multiWildcards = isc.isA.Array(dataSource.patternMultiWildcard) ? dataSource.patternMultiWildcard : [dataSource.patternMultiWildcard || "*"];
        var escapeFound = false;
        var searchExpression = "";
        var value = criterionValues.value;

        // Small state machine to go through the value string, find escape characters as well
        // as wildcard characters and build a regular expression.
        for(var i = 0; i < value.length; i++) {
            // If the character matches the patternEscapeChar on the DataSource, set the engine
            // to ESCAPE state and continue with the next character.
            if (value[i] === dataSource.patternEscapeChar) {
                escapeFound = true;
                continue;
            }

            // If the previous character was an escape character then we need to make sure
            // that this character will treated as a string literal by the regex engine.
            if (escapeFound) {
                // We do this by checking if its a special regex character and if it is we
                // prefix it with the regex escape character.
                searchExpression += value[i].replace(/([.*+?^=!:${}()|\[\]\/\\])/g, "\\$1");

                // Go back to normal state now that the escape has been dealt with.
                escapeFound = false;
            } else {
                // We know that the previous character was not an escape character to lets
                // carry on.

                // If we've come across a multi wildcard, lets add regex .* and continue.
                if (multiWildcards.contains(value[i])) {
                    searchExpression += ".*";
                    continue;
                }

                // If we've come across a single wildcard, lets add regex . and continue.
                if (singleWildcards.contains(value[i])) {
                    searchExpression += ".";
                    continue;
                }

                searchExpression += value[i];
            }
        }

        criterionValues.value = "^" + searchExpression + "$";

        // Now lets pass this over to the regexpCheck method and let it do its thing.
        return regexpCheck.apply(this, [fieldName, fieldValue, criterionValues, dataSource, isDateField]);
    };

    var contains = function (fieldName, fieldValue, criterionValues, dataSource, isDateField) {
        // We don't support strict SQL mode for this operator - it raises too many hard problems
        // on the server side.
        var value = criterionValues.value || [];

        // Convert a non-null single value into an Array containing that value
        if (!isc.isAn.Array(value)) {
            value = [ value ];
        }

        if (isc.isA.Date(fieldValue)) {
            for (var i = 0; i < value.length; i++) {
                var isDate = isc.isA.Date(value[i]);

                // convert relative dates as necessary
                if (isDateField && !isDate && isc.DateUtil.isRelativeDate(value[i])) {
                    value[i] = isc.DateUtil.getAbsoluteDate(value[i]);
                }

                if (isc.isA.Date(value[i]) && dataSource.compareDates(value[i], fieldValue, fieldName) == 0) {
                    return true;
                }
            }
        } else {
            return value.contains(fieldValue);
        }
    };

    // With the new operators condition api this could probably be merged with the equality method.
    var fieldValueCheck = function (fieldName, fieldValue, criterionValues, dataSource, isDateField) {
        // Include all records if the "other" field name is null
        if (criterionValues.value == null) {
            return true;
        }

        // We do not support strict SQL mode for this operator, because different databases behave differently
        // if the two fields are of different types - we can't match a behavior that varies by database.
        var fieldsMatch = false;

        if (isc.isAn.Array(fieldValue) && isc.isAn.Array(criterionValues.otherValue)) {
            fieldsMatch = fieldValue.equals(criterionValues.otherValue);
        } else {
            fieldsMatch = (criterionValues.otherValue == fieldValue);
        }

        // Dates aren't equal in JS unless they are the same object, so we need a special compare
        if (isc.isA.Date(criterionValues.otherValue) && isc.isA.Date(fieldValue)) {
            fieldsMatch = (dataSource.compareDates(criterionValues.otherValue, fieldValue, fieldName, criterionValues.value) == 0);
        }

        return fieldsMatch;
    };

    var fieldStringComparison = function (fieldName, fieldValue, criterionValues, dataSource, isDateField) {
        // Include all records if the "other" field name is null
        if (criterionValues.value == null) {
            return true;
        }

        criterionValues.value = criterionValues.otherValue;

        // Calling another operator condition, we need to use apply here in order to fool the scope.
        // This is because this.negate needs to be the correct one according to this operator.
        return stringComparison.apply(this, [fieldName, fieldValue, criterionValues, dataSource, isDateField]);
    };

    var fieldRangeCheck = function (fieldName, fieldValue, criterionValues, dataSource, isDateField) {
        // Include all records if the "other" field name is null
        if (criterionValues.value == null) {
            return true;
        }

        criterionValues.value = criterionValues.otherValue;

        // "tested" is the value of "otherField".  If it is null or undefined, the rangeCheck()
        // method is going to assume that means "no bounds".  That's OK for a straight
        // range check - if you specify "lessThan" and then provide no value to compare
        // against, it seems more likely that you meant "don't check" rather than
        // "exclude everything".  But in the case of a field comparison, the compare value
        // is different for every row, so it seems much more likely that you would want a
        // null value to be considered "low" - so in a "lessThanField" test, you would expect
        // a row to be excluded if the primary field was non-null and the otherField was null.
        // The final param to the rangeCheck() call switches off this "no bounds" behavior
        return rangeCheck.apply(this, [fieldName, fieldValue, criterionValues, dataSource, isDateField, true]);
    };

    var booleanOperator = function (fieldName, fieldValue, criterionValues, dataSource, isDateField) {
        var criterion = criterionValues.criterion;
        var record = criterionValues.record;
        var returnVal;

        if (!criterion.criteria) {
            // This can legitimately happen if the user proceeds with an "empty" FilterBuilder
            // (ie, one they haven't set any values on).  FilterBuilder strips out all the
            // empty conditions and returns an empty array; something further downstream is
            // throwing that empty array away, so by the time we get here criteria is undef.
            // So we'll just set an undef criteria to the empty array and let it drop through
            criterion.criteria = [];
        }
        if (!isc.isAn.Array(criterion.criteria)) {
            isc.logWarn("AdvancedCriteria: found boolean operator where subcriteria was not " +
                "an array.  Subcriteria was: " + isc.echoFull(criterion.criteria));
            return false;
        }

        // Because we now need to match SQL null-handling behavior, we have to return false even
        // for negated clauses such as not(value equals null). Therefore, we can't just return false
        // if we find a null - we need to return false in the normal case, true where we're nested
        // inside a "not" clause, and then toggle back and forth between true and false as we progress
        // through multiple levels of nested "not" clauses.

        if (this.isNot) {
            dataSource._withinLogicalNot = !dataSource._withinLogicalNot;
        }

        for (var idx = 0; idx < criterion.criteria.length; idx++) {
            var bool = dataSource.evaluateCriterion(record, criterion.criteria[idx]);

            if (this.isAnd && !bool) {
                returnVal = false;
            }

            if (this.isNot && bool) {
                returnVal = false;
            }

            if (this.isOr && bool) {
                returnVal = true;
            }

            if (returnVal != null) {
                break;
            }
        }

        // If we get here and returnVal is undefined, "and" or "not" have succeeded, "or" has failed
        if (returnVal == null) {
            returnVal = !this.isOr;
        }

        // Put "within logical not" status back the way it was
        if (this.isNot) {
            dataSource._withinLogicalNot = !dataSource._withinLogicalNot;
        }

        return returnVal;
    };

    // Operator criterion comparators
    var booleanOperatorComp =
        function (newCriterion, oldCriterion, operator, ds) {
            if (!oldCriterion.criteria) oldCriterion.criteria = [];
            if (!isc.isAn.Array(oldCriterion.criteria)) {
                isc.logWarn("AdvancedCriteria: boolean compareCriteria found " +
                            "where old subcriteria was not an array");
                return -1;
            }
            if (!newCriterion.criteria) newCriterion.criteria = [];
            if (!isc.isAn.Array(newCriterion.criteria)) {
                isc.logWarn("AdvancedCriteria: boolean compareCriteria found " +
                            "where new subcriteria was not an array");
                return -1;
            }
            var rtn,
                thisLevelRtn = 0,
                oldCount = oldCriterion.criteria.length,
                newCount = newCriterion.criteria.length;

            if (newCount > oldCount && operator.isOr) {
                // We can indicate less restrictive right now without having to check the
                // sub-criteria - there's an extra "or" condition
                return -1;
            }

            var oldWork = isc.clone(oldCriterion.criteria);
            var newWork = isc.clone(newCriterion.criteria);

            

            for (var i = 0; i < oldCount; i++) {
                var oldSubCriterion = oldWork[i];
                var newSubCriterion = i > newCount ? null : newWork[i];
                if (!newSubCriterion ||
                    (newSubCriterion && newSubCriterion.fieldName != oldSubCriterion.fieldName ||
                    newSubCriterion.operator  != oldSubCriterion.operator  ||
                    newSubCriterion.processed == true)) {
                    newSubCriterion = null;
                    for (var j = 0; j < newCount; j++) {
                        if (newWork[j].processed) continue;
                        if (newWork[j].fieldName == oldSubCriterion.fieldName &&
                            newWork[j].operator  == oldSubCriterion.operator ) {
                            newSubCriterion = newWork[j];
                            break;
                        }
                    }
                }

                if (newSubCriterion && oldSubCriterion) {
                    newSubCriterion.processed = true;
                    rtn = ds.compareAdvancedCriteria(newSubCriterion, oldSubCriterion);
                } else {
                    if (oldSubCriterion && !newSubCriterion) {
                        // Fewer criteria
                        if (operator.isOr) rtn = 1;
                        if (operator.isAnd) rtn = -1;
                        if (operator.isNot) rtn = -1;
                    }
                }
                if (operator.isAnd && rtn == -1) return -1;
                if (operator.isOr && rtn == -1) return -1;
                if (operator.isNot && rtn == 1) return -1;
                // If any sub-criterion compare comes back as 1 (-1 for NOT), then we should
                // return 1; if they are all 0 we should return 0.
                if (rtn != 0) thisLevelRtn = 1;
            }

            for (var i = 0; i < newCount; i++ ) {
                if (!newWork[i].processed) {
                    // Either there are more new criteria than old, or we've found at least one
                    // that could not be matched.
                    if (operator.isOr) return -1;
                    if (operator.isAnd) return 1;
                    if (operator.isNot) return -1;
                }
            }

            // If we get here, we can indicate identical / more restrictive
            return thisLevelRtn;
        };

    var equalityComp =
        function (newCriterion, oldCriterion, operator, ds) {
            // There's no way an equality check can be more restrictive - either it's identical,
            // or it's less restrictive
            if (newCriterion.fieldName == oldCriterion.fieldName) {
                var eq = ds.compareValues(newCriterion.value, oldCriterion.value,
                                newCriterion.fieldName) == 0;
                if (eq) {
                    return 0;
                } else {
                    return -1;
                }
            } else {
                return -1;
            }
        };

    var rangeCheckComp = function (newCriterion, oldCriterion, operator, ds) {
        // We can return 1 as follows:
        // 1 greaterThan/greaterOrEqual. Return 1 if the new value is larger than the old one.
        // 2 lessThan/lessThanOrEqual.   Return 1 if the new value is smaller than the old one.
        // 3 between/betweenInclusive.   Return 1 if the new start value is larger than the old
        //                               start value but not larger than the old end value,
        //                               OR the new end value is smaller than the old end value
        //                               but not smaller than the old start value.
        var fieldName = newCriterion.fieldName;
        if (newCriterion.fieldName == oldCriterion.fieldName) {
            if (operator.upperBounds && operator.lowerBounds) {
                if (ds.compareValues(newCriterion.start, oldCriterion.start, fieldName) == 0) {
                    if (ds.compareValues(newCriterion.end, oldCriterion.end, fieldName) == 0) {
                        return 0;
                    }
                }
            } else {
                if (ds.compareValues(newCriterion.value, oldCriterion.value, fieldName) == 0) {
                    return 0;
                }
            }

            var newStart = newCriterion.start == null ? newCriterion.value : newCriterion.start,
                oldStart = oldCriterion.start == null ? oldCriterion.value : oldCriterion.start,
                newEnd   = newCriterion.start == null ? newCriterion.value : newCriterion.end,
                oldEnd   = oldCriterion.start == null ? oldCriterion.value : oldCriterion.end;

            var isDate,
                isnumber;

            var oldLowerBounds = true,
                oldUpperBounds = true,
                newLowerBounds = true,
                newUpperBounds = true;

             // See the IDoc in the rangeCheck() function for a discussion of why we're doing this
            if (oldStart == null) oldLowerBounds = false;
            if (oldEnd == null) oldUpperBounds = false;
            if (newStart == null) newLowerBounds = false;
            if (newEnd == null) newUpperBounds = false;

            if (operator.lowerBounds && !operator.upperBounds &&
                    !newLowerBounds && !oldLowerBounds) {
                return 0;
            }

            if (operator.lowerBounds && !operator.upperBounds) {
                if (newLowerBounds && !oldLowerBounds) {
                    return 1;
                }
                if (oldLowerBounds && !newLowerBounds) {
                    return -1;
                }
                var rslt = ds.compareValues(newStart, oldStart, fieldName);
                if (rslt == -1) {
                    return 1;
                } else if (rslt == 0) {
                    return 0;
                }
            }

            if (operator.upperBounds && !operator.lowerBounds &&
                    !newUpperBounds && !oldUpperBounds) {
                return 0;
            }


            if (operator.upperBounds && !operator.lowerBounds) {
                if (newUpperBounds && !oldUpperBounds) {
                    return 1;
                }
                if (oldUpperBounds && !newUpperBounds) {
                    return -1;
                }
                var rslt = ds.compareValues(newStart, oldStart, fieldName);
                if (rslt == 1) {
                    return 1;
                } else if (rslt == 0) {
                    return 0;
                }
            }

            if (operator.lowerBounds && operator.upperBounds) {
                // Technically, checking for >= and <= here is not sufficient to infer more
                // restriction; however, it IS enough to infer either more restriction or an
                // identical pair of filters, and since we won't get this far if the filters
                // are identical, it's safe to return 1.
                if (ds.compareValues(newStart, oldStart, fieldName) <= 0 &&
                    ds.compareValues(newStart, oldEnd, fieldName) >= 0 &&
                    ds.compareValues(newEnd, oldEnd, fieldName) >= 0 &&
                    ds.compareValues(newEnd, oldStart, fieldName) <= 0)
                {
                    return 1;
                }
                if ((newLowerBounds && !oldLowerBounds) ||
                    (newUpperBounds && !oldUpperBounds)) {
                    return 1;
                }
                if (!newLowerBounds && !oldLowerBounds &&
                    !newUpperBounds && !oldLowerBounds) {
                    return 0;
                }
            }
        }
        return -1;
    };

    var stringComparisonComp =
        function (newCriterion, oldCriterion, operator) {
            // We can return 1 as  follows:
            // 1 startsWith. Return 1 if the new value startsWith the old value AND is longer.
            //               Obey the caseInsensitive flag.
            //               If negate == true, the test is reversed - ie, old value must
            //               startWith new value, and new value must be shorter.
            // 2 endsWith.   Return 1 if the new value endsWith the old value AND is longer.
            //               Obey the caseInsensitive flag.
            //               If negate == true, the test is reversed - ie, old value must
            //               endWith new value, and new value must be shorter.
            // 3 contains.   Return 1 if the old value is a proper substring of the new value (ie,
            //               new value both contains and is longer than old value). Obey the
            //               caseInsensitive flag. If negate == true, the new field must be a
            //               substring of the old one.
            // 4 equals.     Return 0 if the new value exactly matches the old one.  This
            //               comparison is actually only used when in case-insensitive mode;
            //               obviously, we simply compare for equality otherwise.

            var oldVal = oldCriterion.value;
            var newVal = newCriterion.value;

            // Convert numbers to strings - other than that, reject anything that isn't a string
            if (isc.isA.Number(oldVal)) oldVal = "" + oldVal;
            if (isc.isA.Number(newVal)) newVal = "" + newVal;

            if (!isc.isA.String(oldVal) || !isc.isA.String(newVal)) return -1;

            if (operator.caseInsensitive) {
                oldVal = oldVal.toLowerCase();
                newVal = newVal.toLowerCase();
            }

            if (newCriterion.fieldName == oldCriterion.fieldName
                    && newCriterion.value == oldCriterion.value
                    && !operator.equals)
            {
                return 0;
            }

            if (operator.equals) {
                return oldVal == newVal ? 0 : -1;
            }

            if (operator.startsWith && !operator.negate &&
                    newVal.length > oldVal.length && isc.startsWith(newVal, oldVal))
            {
                return 1;
            }


            if (operator.startsWith && operator.negate &&
                    oldVal.length > newVal.length && isc.startsWith(oldVal, newVal))
            {
                return 1;
            }

            if (operator.endsWith && !operator.negate &&
                    newVal.length > oldVal.length && isc.endsWith(newVal, oldVal))
            {
                return 1;
            }

            if (operator.endsWith && operator.negate &&
                    oldVal.length > newVal.length && isc.endsWith(oldVal, newVal))
            {
                return 1;
            }

            if (!operator.startsWith && !operator.endsWith && !operator.negate &&
                    newVal.length > oldVal.length && isc.contains(newVal, oldVal))
            {
                return 1;
            }

            if (!operator.startsWith && !operator.endsWith && operator.negate &&
                    oldVal.length > newVal.length && isc.contains(oldVal, newVal))
            {
                return 1;
            }

            return -1;
        };

    var nullCheckComp =
        function (newCriterion, oldCriterion, operator) {
            if (newCriterion.fieldName == oldCriterion.fieldName)  return 0;
            else return -1;
        };

    var regexpCheckComp =
        function (newCriterion, oldCriterion, operator) {
            // We can't know whether the new regexp is more or less restrictive - either they
            // match, or we indicate less restrictive
            if (newCriterion.value == oldCriterion.value &&
                      newCriterion.fieldName == oldCriterion.fieldName) {
                return 0;
            } else {
                return -1;
            }
        };

    var setMembershipComp =
        function (newCriterion, oldCriterion, operator) {
            // We can return 1 if the new set is a proper subset of the old one
            // (if negate == true, the old set must be a proper subset of the new one)
            if (newCriterion.fieldName == oldCriterion.fieldName) {

                // We can only sensibly compare Arrays
                if (!isc.isAn.Array(oldCriterion.value) || !isc.isAn.Array(newCriterion.value)) {
                    return -1;
                }

                if (newCriterion.value.equals(oldCriterion.value)) {
                    return 0;
                }

                if (!operator.negate && oldCriterion.value.containsAll(newCriterion.value)) {
                    return 1;
                }

                if (operator.negate && newCriterion.value.containsAll(oldCriterion.value)) {
                    return 1;
                }
            }

            return -1;
        };

    var fieldValueCheckComp =
        function (newCriterion, oldCriterion, operator) {
            // There's no way this check can be more restrictive - either it's the same
            // or it's different, and if it's different we have to go back to the server.
            if (newCriterion.value == oldCriterion.value &&
                      newCriterion.fieldName == oldCriterion.fieldName) {
                return 0;
            } else {
                return -1;
            }
        };


    var builtinOps = [
    {
        ID: "equals",
        titleProperty: "equalsTitle",
        negate: false,
        valueType: "fieldType",
        condition: equality,
        compareCriteria: equalityComp,
        symbol: "==",
        wildCard: "*",
        canNormalize: true,
        getCriterion : function (fieldName, item) {
            var result = { fieldName: fieldName, operator: this.ID };

            if (isc.isA.RelativeDateItem(item))
                result.value = item.getRelativeDate() || item.getValue();
            else result.value = item.getValue();

            return result;
        }
    },
    
    {
        ID: "notEqual",
        titleProperty: "notEqualTitle",
        negate: true,
        valueType: "fieldType",
        condition: equality,
        compareCriteria: equalityComp,
        symbol: "!",
        wildCard: "*",
        canNormalize: true,
        getCriterion : function (fieldName, item) {
            var result = { fieldName: fieldName, operator: this.ID };

            if (isc.isA.RelativeDateItem(item))
                result.value = item.getRelativeDate() || item.getValue();
            else result.value = item.getValue();

            return result;
        }
    },
    {
        ID: "greaterThan",
        titleProperty: "greaterThanTitle",
        lowerBounds: true,
        valueType: "fieldType",
        condition: rangeCheck,
        compareCriteria: rangeCheckComp,
        symbol: ">",
        getCriterion : function (fieldName, item) {
            var result = { fieldName: fieldName, operator: this.ID };

            if (isc.isA.RelativeDateItem(item))
                result.value = item.getRelativeDate() || item.getValue();
            else result.value = item.getValue();

            return result;
        }
    },
    {
        ID: "lessThan",
        titleProperty: "lessThanTitle",
        upperBounds: true,
        valueType: "fieldType",
        condition: rangeCheck,
        compareCriteria: rangeCheckComp,
        symbol: "<",
        getCriterion : function (fieldName, item) {
            var result = { fieldName: fieldName, operator: this.ID };

            if (isc.isA.RelativeDateItem(item)) 
                result.value = item.getRelativeDate() || item.getValue();
            else result.value = item.getValue();

            return result;
        }
    },
    {
        ID: "greaterOrEqual",
        titleProperty: "greaterOrEqualTitle",
        lowerBounds: true,
        inclusive: true,
        valueType: "fieldType",
        condition: rangeCheck,
        compareCriteria: rangeCheckComp,
        symbol: ">=",
        getCriterion : function (fieldName, item) {
            var result = { fieldName: fieldName, operator: this.ID };

            if (isc.isA.RelativeDateItem(item))
                result.value = item.getRelativeDate() || item.getValue();
            else result.value = item.getValue();

            return result;
        }
    },
    {
        ID: "lessOrEqual",
        titleProperty: "lessOrEqualTitle",
        upperBounds: true,
        inclusive: true,
        valueType: "fieldType",
        condition: rangeCheck,
        compareCriteria: rangeCheckComp,
        symbol: "<=",
        getCriterion : function (fieldName, item) {
            var result = { fieldName: fieldName, operator: this.ID };

            if (isc.isA.RelativeDateItem(item)) 
                result.value = item.getRelativeDate() || item.getValue();
            else result.value = item.getValue();

            return result;
        }
    },
    {
        ID: "between",
        titleProperty: "betweenTitle",
        lowerBounds: true,
        upperBounds: true,
        hidden:true,
        valueType: "valueRange",
        condition: rangeCheck,
        compareCriteria: rangeCheckComp,
        getCriterion : function (fieldName, item) {
            var result = { fieldName: fieldName, operator: this.ID };
            if (isc.isA.RelativeDateItem(item)) 
                result.value = item.getRelativeDate() || item.getValue();
            else result.value = item.getValue();

            return result;
        }
    },
    {
        ID: "iBetween",
        titleProperty: "iBetweenTitle",
        lowerBounds: true,
        upperBounds: true,
        hidden:true,
        valueType: "valueRange",
        condition: rangeCheck,
        caseInsensitive: true,
        compareCriteria: rangeCheckComp,
        getCriterion : function (fieldName, item) {
            var result = { fieldName: fieldName, operator: this.ID };
            if (isc.isA.RelativeDateItem(item)) 
                result.value = item.getRelativeDate() || item.getValue();
            else result.value = item.getValue();

            return result;
        }
    },
    {
        ID: "betweenInclusive",
        titleProperty: "betweenInclusiveTitle",
        lowerBounds: true,
        upperBounds: true,
        valueType: "valueRange",
        inclusive: true,
        condition: rangeCheck,
        compareCriteria: rangeCheckComp,
        symbol: "...",
        getCriterion : function (fieldName, item) {
            var result = { fieldName: fieldName, operator: this.ID };
            if (isc.isA.RelativeDateItem(item))
                result.value = item.getRelativeDate() || item.getValue();
            else result.value = item.getValue();

            return result;
        }
    },
    {
        ID: "iBetweenInclusive",
        titleProperty: "iBetweenInclusiveTitle",
        lowerBounds: true,
        upperBounds: true,
        valueType: "valueRange",
        inclusive: true,
        condition: rangeCheck,
        compareCriteria: rangeCheckComp,
        symbol: "...",
        caseInsensitive: true,
        getCriterion : function (fieldName, item) {
            var result = { fieldName: fieldName, operator: this.ID };
            if (isc.isA.RelativeDateItem(item))
                result.value = item.getRelativeDate() || item.getValue();
            else result.value = item.getValue();

            return result;
        }
    },
    {
        ID: "iEquals",
        titleProperty: "iEqualsTitle",
        equals: true,
        caseInsensitive: true,
        valueType: "fieldType",
        condition: stringComparison,
        symbol: "==",
        wildCard: "*",
        canNormalize: true,
        compareCriteria: stringComparisonComp
    },
    {
        ID: "iContains",
        titleProperty: "iContainsTitle",
        caseInsensitive: true,
        valueType: "fieldType",
        condition: stringComparison,
        symbol: "~",
        canNormalize: true,
        compareCriteria: stringComparisonComp
    },
    {
        ID: "iStartsWith",
        titleProperty: "iStartsWithTitle",
        startsWith: true,
        caseInsensitive: true,
        valueType: "fieldType",
        condition: stringComparison,
        symbol: "^",
        canNormalize: true,
        compareCriteria: stringComparisonComp
    },
    {
        ID: "iEndsWith",
        titleProperty: "iEndsWithTitle",
        endsWith: true,
        caseInsensitive: true,
        valueType: "fieldType",
        condition: stringComparison,
        symbol: "|",
        canNormalize: true,
        compareCriteria: stringComparisonComp
    },
    {
        ID: "contains",
        titleProperty: "containsTitle",
        hidden:true,
        valueType: "fieldType",
        condition: stringComparison,
        symbol: "~",
        canNormalize: true,
        compareCriteria: stringComparisonComp
    },
    {
        ID: "startsWith",
        titleProperty: "startsWithTitle",
        startsWith: true,
        hidden:true,
        valueType: "fieldType",
        condition: stringComparison,
        symbol: "^",
        canNormalize: true,
        compareCriteria: stringComparisonComp
    },
    {
        ID: "endsWith",
        titleProperty: "endsWithTitle",
        endsWith: true,
        hidden:true,
        valueType: "fieldType",
        condition: stringComparison,
        symbol: "|",
        canNormalize: true,
        compareCriteria: stringComparisonComp
    },
    {
        ID: "iNotEqual",
        titleProperty: "iNotEqualTitle",
        caseInsensitive: true,
        equals: true,
        negate: true,
        valueType: "fieldType",
        condition: stringComparison,
        symbol: "!",
        wildCard: "*",
        canNormalize: true,
        compareCriteria: stringComparisonComp
    },
    {
        ID: "iNotContains",
        titleProperty: "iNotContainsTitle",
        caseInsensitive: true,
        negate: true,
        valueType: "fieldType",
        condition: stringComparison,
        symbol: "!~",
        canNormalize: true,
        compareCriteria: stringComparisonComp
    },
    {
        ID: "iNotStartsWith",
        titleProperty: "iNotStartsWithTitle",
        startsWith: true,
        caseInsensitive: true,
        negate: true,
        valueType: "fieldType",
        condition: stringComparison,
        symbol: "!^",
        canNormalize: true,
        compareCriteria: stringComparisonComp
    },
    {
        ID: "iNotEndsWith",
        titleProperty: "iNotEndsWithTitle",
        endsWith: true,
        caseInsensitive: true,
        negate: true,
        valueType: "fieldType",
        condition: stringComparison,
        symbol: "!@",
        canNormalize: true,
        compareCriteria: stringComparisonComp
    },
    {
        ID: "notContains",
        titleProperty: "notContainsTitle",
        negate: true,
        hidden:true,
        valueType: "fieldType",
        condition: stringComparison,
        symbol: "!~",
        canNormalize: true,
        compareCriteria: stringComparisonComp
    },
    {
        ID: "notStartsWith",
        titleProperty: "notStartsWithTitle",
        startsWith: true,
        negate: true,
        hidden:true,
        valueType: "fieldType",
        condition: stringComparison,
        symbol: "!^",
        canNormalize: true,
        compareCriteria: stringComparisonComp
    },
    {
        ID: "notEndsWith",
        titleProperty: "notEndsWithTitle",
        endsWith: true,
        negate: true,
        hidden:true,
        valueType: "fieldType",
        condition: stringComparison,
        symbol: "!@",
        canNormalize: true,
        compareCriteria: stringComparisonComp
    },
    {
        ID: "isNull",
        titleProperty: "isNullTitle",
        valueType: "none",
        condition: nullCheck,
        symbol: "#",
        compareCriteria: nullCheckComp
    },
    {
        ID: "notNull",
        titleProperty: "notNullTitle",
        negate: true,
        valueType: "none",
        condition: nullCheck,
        symbol: "!#",
        compareCriteria: nullCheckComp
    },
    {
        ID: "regexp",
        titleProperty: "regexpTitle",
        hidden: true, 
        valueType: "custom",
        condition: regexpCheck,
        symbol: "/regex/",
        compareCriteria: regexpCheckComp
    },
    {
        ID: "iregexp",
        titleProperty: "iregexpTitle",
        hidden: true,
        caseInsensitive: true,
        valueType: "custom",
        condition: regexpCheck,
        symbol: "/regex/",
        compareCriteria: regexpCheckComp
    },
    {
        ID: "inSet",
        titleProperty: "inSetTitle",
        hidden: true,
        valueType: "valueSet",
        condition: contains,
        compareCriteria: setMembershipComp,
        symbol: "=(",
        closingSymbol: ")",
        valueSeparator: "|",
        processValue : function (value, ds) {
            return value.split(this.valueSeparator);
        }
    },
    {
        ID: "notInSet",
        titleProperty: "notInSetTitle",
        negate: true,
        hidden: true,
        valueType: "valueSet",
        condition: contains,
        compareCriteria: setMembershipComp,
        symbol: "!=(",
        closingSymbol: ")",
        valueSeparator: "|",
        processValue : function (value, ds) {
            return value.split(this.valueSeparator);
        }
    },
    {
        ID: "equalsField",
        titleProperty: "equalsFieldTitle",
        valueType: "fieldName",
        condition: fieldValueCheck,
        symbol: "=.",
        canNormalize: true,
        compareCriteria: fieldValueCheckComp,
        processValue : function (value, ds) {
            if (!ds) return value;
            
            var field = ds.getField(value);
            
            if (field) return value;
            
            field = ds.getFieldByTitle(value);
            
            if (field) return field.name;
            
            return null;
        }
    },
    {
        ID: "notEqualField",
        titleProperty: "notEqualFieldTitle",
        negate: true,
        valueType: "fieldName",
        condition: fieldValueCheck,
        canNormalize: true,
        compareCriteria: fieldValueCheckComp
    },
    {
        ID: "iEqualsField",
        titleProperty: "iEqualsFieldTitle",
        equals: true,
        caseInsensitive:true,
        valueType: "fieldName",
        condition: fieldStringComparison,
        canNormalize: true,
        compareCriteria: fieldValueCheckComp
    },
    {
        ID: "iNotEqualField",
        titleProperty: "iNotEqualFieldTitle",
        equals: true,
        caseInsensitive:true,
        negate: true,
        valueType: "fieldName",
        condition: fieldStringComparison,
        canNormalize: true,
        compareCriteria: fieldValueCheckComp
    },
    {
        ID: "greaterThanField",
        titleProperty: "greaterThanFieldTitle",
        lowerBounds: true,
        valueType: "fieldName",
        condition: fieldRangeCheck,
        compareCriteria: fieldValueCheckComp
    },
    {
        ID: "lessThanField",
        titleProperty: "lessThanFieldTitle",
        upperBounds: true,
        valueType: "fieldName",
        condition: fieldRangeCheck,
        compareCriteria: fieldValueCheckComp
    },
    {
        ID: "greaterOrEqualField",
        titleProperty: "greaterOrEqualFieldTitle",
        lowerBounds: true,
        inclusive: true,
        valueType: "fieldName",
        condition: fieldRangeCheck,
        compareCriteria: fieldValueCheckComp
    },
    {
        ID: "lessOrEqualField",
        titleProperty: "lessOrEqualFieldTitle",
        upperBounds: true,
        inclusive: true,
        valueType: "fieldName",
        condition: fieldRangeCheck,
        compareCriteria: fieldValueCheckComp
    },
    {
        ID: "containsField",
        titleProperty: "containsFieldTitle",
        hidden:true,
        valueType: "fieldName",
        condition: fieldStringComparison,
        canNormalize: true,
        compareCriteria: fieldValueCheckComp
    },
    {
        ID: "startsWithField",
        titleProperty: "startsWithTitleField",
        startsWith: true,
        hidden:true,
        valueType: "fieldName",
        condition: fieldStringComparison,
        canNormalize: true,
        compareCriteria: fieldValueCheckComp
    },
    {
        ID: "endsWithField",
        titleProperty: "endsWithTitleField",
        endsWith: true,
        hidden:true,
        valueType: "fieldName",
        condition: fieldStringComparison,
        canNormalize: true,
        compareCriteria: fieldValueCheckComp
    },
    {
        ID: "iContainsField",
        titleProperty: "iContainsFieldTitle",
        hidden:true,
        caseInsensitive:true,
        valueType: "fieldName",
        condition: fieldStringComparison,
        canNormalize: true,
        compareCriteria: fieldValueCheckComp
    },
    {
        ID: "iStartsWithField",
        titleProperty: "iStartsWithTitleField",
        startsWith: true,
        hidden:true,
        caseInsensitive:true,
        valueType: "fieldName",
        condition: fieldStringComparison,
        canNormalize: true,
        compareCriteria: fieldValueCheckComp
    },
    {
        ID: "iEndsWithField",
        titleProperty: "iEndsWithTitleField",
        endsWith: true,
        hidden:true,
        caseInsensitive:true,
        valueType: "fieldName",
        condition: fieldStringComparison,
        canNormalize: true,
        compareCriteria: fieldValueCheckComp
    },
    {
        ID: "notContainsField",
        titleProperty: "notContainsFieldTitle",
        hidden:true,
        negate: true,
        valueType: "fieldName",
        condition: fieldStringComparison,
        canNormalize: true,
        compareCriteria: fieldValueCheckComp
    },
    {
        ID: "notStartsWithField",
        titleProperty: "notStartsWithTitleField",
        startsWith: true,
        hidden:true,
        negate: true,
        valueType: "fieldName",
        condition: fieldStringComparison,
        canNormalize: true,
        compareCriteria: fieldValueCheckComp
    },
    {
        ID: "notEndsWithField",
        titleProperty: "notEndsWithTitleField",
        endsWith: true,
        hidden:true,
        negate: true,
        valueType: "fieldName",
        condition: fieldStringComparison,
        canNormalize: true,
        compareCriteria: fieldValueCheckComp
    },
    {
        ID: "iNotContainsField",
        titleProperty: "iNotContainsFieldTitle",
        hidden:true,
        caseInsensitive:true,
        negate: true,
        valueType: "fieldName",
        condition: fieldStringComparison,
        canNormalize: true,
        compareCriteria: fieldValueCheckComp
    },
    {
        ID: "iNotStartsWithField",
        titleProperty: "iNotStartsWithTitleField",
        startsWith: true,
        hidden:true,
        caseInsensitive:true,
        negate: true,
        valueType: "fieldName",
        condition: fieldStringComparison,
        canNormalize: true,
        compareCriteria: fieldValueCheckComp
    },
    {
        ID: "iNotEndsWithField",
        titleProperty: "iNotEndsWithTitleField",
        endsWith: true,
        hidden:true,
        caseInsensitive:true,
        negate: true,
        valueType: "fieldName",
        condition: fieldStringComparison,
        canNormalize: true,
        compareCriteria: fieldValueCheckComp
    },
    {
        ID: "and",
        titleProperty: "andTitle",
        isAnd: true,
        valueType: "criteria",
        condition: booleanOperator,
        symbol: " and ",
        compareCriteria: booleanOperatorComp
    },
    {
        ID: "not",
        titleProperty: "notTitle",
        isNot: true,
        valueType: "criteria",
        condition: booleanOperator,
        compareCriteria: booleanOperatorComp
    },
    {
        ID: "or",
        titleProperty: "orTitle",
        isOr: true,
        valueType: "criteria",
        condition: booleanOperator,
        symbol: " or ",
        compareCriteria: booleanOperatorComp
    }
    ];

    for (var idx = 0; idx < builtinOps.length; idx++) {
        isc.DataSource.addSearchOperator(builtinOps[idx]);
    }

    // Create default typeOperators
    isc.DataSource.setTypeOperators(null, ["equals", "notEqual", "lessThan", "greaterThan",
                                           "lessOrEqual", "greaterOrEqual", "between",
                                           "betweenInclusive", "isNull", "notNull", "inSet",
                                           "notInSet", "equalsField", "notEqualField",
                                           "greaterThanField", "lessThanField",
                                           "greaterOrEqualField", "lessOrEqualField",
                                           "and", "or", "not", "inSet", "notInSet"]);

    // these are in addition to the base operators
    isc.DataSource.setTypeOperators("text", ["regexp", "iregexp", "contains", "startsWith",
                                               "endsWith", "iEquals", "iNotEqual",
                                               "iBetween", "iBetweenInclusive",
                                               "iContains", "iStartsWith", "iEndsWith",
                                               "notContains", "notStartsWith", "notEndsWith", 
                                               "iNotContains", "iNotStartsWith", "iNotEndsWith",
                                               "containsField", "startsWithField", "endsWithField",
                                               "iEqualsField", "iNotEqualField",
                                               "iContainsField", "iStartsWithField", "iEndsWithField",
                                               "notContainsField", "notStartsWithField", "notEndsWithField",
                                               "iNotContainsField", "iNotStartsWithField", "iNotEndsWithField"]);
    
    isc.DataSource.setTypeOperators("integer", ["iContains", "iStartsWith", "iEndsWith",
                                               "iNotContains", "iNotStartsWith", "iNotEndsWith", 
                                               "iEqualsField", "iNotEqualField",
                                               "containsField", "startsWithField", "endsWithField",
                                               "iContainsField", "iStartsWithField", "iEndsWithField",
                                               "notContainsField", "notStartsWithField", "notEndsWithField",
                                               "iNotContainsField", "iNotStartsWithField", "iNotEndsWithField"]);
    
    isc.DataSource.setTypeOperators("float", ["iContains", "iStartsWith", "iEndsWith",
                                               "iNotContains", "iNotStartsWith", "iNotEndsWith", 
                                               "iEqualsField", "iNotEqualField",
                                               "containsField", "startsWithField", "endsWithField",
                                               "iContainsField", "iStartsWithField", "iEndsWithField",
                                               "notContainsField", "notStartsWithField", "notEndsWithField",
                                               "iNotContainsField", "iNotStartsWithField", "iNotEndsWithField"]);

};

isc._initBuiltInOperators();


// Built-in Schema
// ---------------------------------------------------------------------------------------

// A generic "Object" DataSource is needed so that you can declare that a field is of Object
// type, as opposed to simple type, without having to be more specific.
isc.DataSource.create({ ID:"Object", fields: {}, addGlobalId:false });

isc.DataSource.create({ 
    ID:"ValueMap", 
    addGlobalId:false,
    // prevent clobbering by server definitions
    builtinSchema : true,
    // a valueMap can appear as a simple JS Array, and in this case shouldn't be considered
    // multiple valueMaps
    canBeArrayValued : true, 
    //acceptsSimpleTypes : true,
    fields : {},
    _$ID : "ID", _$id : "id",
    xmlToJS : function (element, context) {
        if (element == null || isc.xml.elementIsNil(element)) return null;

        var children = isc.xml.getElementChildren(element),
            valueMap = isc.xml.getAttributes(element),
            // any attribute is considered to be an id->value mapping
            gotIDMapping = !isc.isAn.emptyObject(valueMap);
        for (var i = 0; i < children.length; i++) {
            var child = children[i],
                ID = child.getAttribute(this._$ID) || child.getAttribute(this._$id),
                value = isc.xml.getElementText(child);
            if (ID != null && value != null) {
                // <value ID="idValue">displayValue</value>
                gotIDMapping = true;
                valueMap[ID] = value;
            } else if (ID != null) {
                // <value ID="idValue">displayValue</value>
                valueMap[ID] = ID;
            } else if (value != null) {
                // <value>onlyValue</value>
                valueMap[value] = value;
            } else {
                // <value/> or <value></value> means the empty value is legal
                valueMap[isc.emptyString] = isc.emptyString;
            }
        }
        // if we got at least one id -> value mapping, return the valueMap as a map
        if (gotIDMapping) return valueMap;
        // otherwise return it as an array
        return isc.getValues(valueMap);
    },
    xmlSerializeFields : function (data, flags, indent) {
        if (data == null || isc.DS.isSimpleTypeValue(data)) {
            return this.Super("xmlSerializeFields", arguments);
        }
    
        var output = isc.SB.create(),
            indent = (indent || "") + "    ";
        if (isc.isAn.Array(data)) {
            for (var i = 0; i < data.length; i++) {
                var value = data[i];
                output.append("\r", indent, "<value>", isc.makeXMLSafe(value), "</value>");
            }
        } else {
            // for objects, output the special valueMap format
            for (var id in data) {
                var value = data[id];
                output.append("\r", indent, "<value id=\"", isc.makeXMLSafe(id), "\">", 
                              isc.makeXMLSafe(value), "</value>");
            }
        }
        return output.toString();
    }
});

// --------------------------------------------------------------------------------------------


isc.ClassFactory.defineInterface("DataModel");


isc.DataModel.addInterfaceMethods({

getDataSource : function () {
    if (isc.isA.String(this.dataSource)) this.dataSource = isc.DS.get(this.dataSource);
    return this.dataSource;
},


getOperationId : function (operationType) {
    var operation = this.getOperation(operationType);
    return operation == null ? null : (isc.isA.String(operation) ? operation : operation.ID);
},


getOperation : function (operationType) {
    var application = isc.rpc.getDefaultApplication(),
        operation,
        operationId;

    // see if an operation for this operation type was explicitly provided
    var operationProperty = operationType + "Operation";
    if (this[operationProperty]) {
        operation = this[operationProperty];
        // support operation spec in object or string form - if in object form, just return it
        if (isc.isAn.Object(operation)) return operation;

        // it's a string - use that as the operationId
        operationId = operation;
    }

    // no operation specified, create an auto-operation using the dataSource provided
    if (operationId == null || isc.isA.String(operationId)) {
        var dataSource = this.getDataSource();
        if (dataSource == null) {
            this.logWarn("can't getOperation for type: " + operationType + 
                         ", no " + operationProperty + " specified, and no dataSource to " +
                         "create an auto-operation");
            return null;
        }
        this.logInfo("creating auto-operation for operationType: " + operationType);
        operation = isc.DataSource.makeDefaultOperation(dataSource, operationType, operationId);
                                                            
        // cache the auto-operation
        operationId = operation.ID;
        this[operationProperty] = operationId;        
    }

    return operation;
}

});

//> @class XJSONDataSource
// A DataSource preconfigured to use the +link{type:RPCTransport,"scriptInclude"} transport
// for cross-domain calls to JSON services, such as those provided by Yahoo!.  
// <P>
// To use this DataSource, provide the URL of the service as +link{DataSource.dataURL}, and
// provide +link{DataSource.fields,fields} that describe the structure of the data you want to
// extract from the service's response.
// <P>
// +link{DataSource.recordXPath} and +link{DataSourceField.valueXPath} can be used to extract
// data from the JSON structure returned by the service.  See
// +link{group:clientDataIntegration,Client-Side Data Integration} for an overview of how to
// control what parts of the JSON structure are included in the +link{DSResponse} object, and
// hence provided to +link{DataBoundComponent}s that are bound to this DataSource.
// <P>
// This XJSONDataSource is really a subclass of DataSource with just a few property settings:
// <pre>
//    dataFormat : "json",
//    dataTransport : "scriptInclude"
//    callbackParam : "callback"
// </pre>
// <P>
// If you are also writing the server side code to respond to requests from this DataSource,
// see the 
// +externalLink{http://developer.yahoo.net/common/json.html#callbackparam,tutorial provided by Yahoo!} 
// for a good overview of how this transport mechanism works.  Note, as indicated in the
// tutorial above, the server is responsible for writing out not just the data, but also a
// JavaScript function call that tells the client that the response has arrived.  The client
// passes the name of the function to call as the "callback" URL parameter.
// <P>
// NOTE: if you use this DataSource to contact Yahoo web services, remember to include
// output=json in the dataURL, as well as a
// +externalLink{http://developer.yahoo.net/,Yahoo developer ID}.
// 
// @treeLocation Client Reference/Data Binding
// @visibility xmlBinding
// @example jsonYahooWebServices
//<
isc.defineClass("XJSONDataSource", "DataSource").addMethods({
    dataFormat : "json",
    dataTransport : "scriptInclude"
});


//> @groupDef clientDataIntegration
// SmartClient supports declarative, XPath-based binding of visual components to any server
// capable of returning XML or JSON responses over HTTP, without the need for the
// +link{group:serverDataIntegration,SmartClient server}.
// <P>
// This approach is called Client-Side Data Integration, which means:
// <ul>
// <li> You +link{group:dataSourceDeclaration,create DataSources} 
// <var class="smartclient">in JavaScript</var>
// <var class="smartgwt">programmatically in Java (with <code>new DataSource()</code>)</var>
// which describe the data to be loaded and manipulated in the user interface. The
// JavaScript that creates these DataSources may be dynamically generated and/or existing
// metadata may be +link{group:metadataImport,imported}.
// <li> You configure DataSources, via property and method overrides, to send appropriate
// HTTP requests to your server, and to parse HTTP responses from your server, in order to
// fulfill the 4 core operations of the +link{group:dataSourceOperations,DataSource Protocol}.
// <li> These DataSources are then bound to 
// +link{dataBoundComponent,databinding-capable UI components}, which can provide a variety of
// complete user interactions (form-based editing, grid-based editing, load on demand, ..)
// based on these 4 core operations
// </ul>
// <P>
// <h4>Approaches and platforms</h4>
// <P>
// <B>REST integration with RestDataSource (preferred)</B>
// <P>
// The +link{RestDataSource} provides a complete XML or JSON-based protocol that supports all
// of the features of SmartClient's databinding layer (data paging, queuing/batching of
// requests for transactions, nested AdvancedCriteria, server-side validation errors, automatic
// cache synchronization, etc).  To use the RestDataSource, simply write server code that can
// parse RestDataSource requests and produce the required responses; example requests and
// responses are +link{RestDataSource,provided}.
// <P>
// The SmartClient public wiki contains examples of integration with
// +externalLink{http://wiki.smartclient.com/display/Main/Integrating+with+ASP.Net+MVC,.NET's ASP.NET MVC}
// as well as 
// +externalLink{http://wiki.smartclient.com/pages/viewpage.action?pageId=1442411,PHP with Doctrine}.
// <P>
// <h4>Consuming Existing XML and JSON formats</h4>
// <p>
// If you have pre-existing XML or JSON formats, SmartClient DataSources can be configured to
// work with them.  However, <b>only use this approach if you are unable to modify a
// pre-existing protocol</b>.  If you have a choice, use RestDataSource.  
// <P>
// In particular, if you are choosing between tools that can automatically generate a REST
// service from an API vs using the pre-built RestDataSource protocol, <b>definitely use
// RestDataSource</b>.  Automatically generated REST interfaces will not handle the needs of a
// modern GUI, such as transactional/batched saves and user-ready validation error messages.
// For a deeper discussion, see 
// +externalLink{http://forums.smartclient.com/showthread.php?t=8159#aExistingRest,this FAQ}.
// <P>
// Specifically for pre-existing WSDL web services, see the discussion of WSDL Integration
// below instead.
// <P>
// To display XML or JSON from a pre-existing service in a visual component such as a ListGrid,
// you bind the component to a +link{DataSource} which provides the
// +link{DataSource.dataURL,URL} of the service, as well as a declaration of how to form inputs
// to the service and how to interpret service responses as DataSource records.
// <P>
// An XPath expression, the +link{attr:operationBinding.recordXPath,recordXPath}, is applied to
// the service response to select the XML elements or JSON objects that should be interpreted
// as DataSource records.  Then, for each field of the DataSource, an optional
// +link{attr:DataSourceField.valueXPath} can be declared which selects the value for the field
// from within each of the XML elements or JSON objects selected by the recordXPath.  If no
// valueXPath is specified, the field name itself is taken as an XPath, which will select the
// same-named subelement or property from the record element or object.
// <P>
// For example, the following code defines a DataSource that a ListGrid could bind to in order
// to display an RSS 2.0 feed.
// <PRE>
//    isc.DataSource.create({
//        dataURL:feedURL,
//        recordXPath:"//item",
//        fields:[
//            { name:"title" },
//            { name:"link" },
//            { name:"description" }
//        ]
//    });
// </PRE>
// A representative slice of an RSS 2.0 feed follows:
// <pre>
//     &lt;?xml version="1.0" encoding="iso-8859-1" ?&gt; 
//     &lt;rss version="2.0"&gt;
//     &lt;channel&gt;
//       &lt;title&gt;feed title&lt;/title&gt; 
//       ...
//       &lt;item&gt;
//         &lt;title&gt;article title&lt;/title&gt; 
//         &lt;link&gt;url of article&lt;/link&gt; 
//         &lt;description&gt;
//            article description
//         &lt;/description&gt; 
//       &lt;/item&gt;
//       &lt;item&gt;
//          ...
// </pre>
// Here, the recordXPath selects a list of &lt;item&gt; elements.  Since the intended values
// for each DataSource field appear as simple subelements of each &lt;item&gt; element (eg
// &lt;description&gt;), the field name is sufficient to select the correct values, and no
// explicit valueXPath needs to be specified.
// <P>
// A running version of this example is available here: +explorerExample{rssFeed}.  Further
// examples of simple XML or JSON data loading using files stored on disk as the
// "service" to contact: the +explorerExample{simpleJSON,Simple JSON} example shows loading
// data from a JSON file into a databound grid, and the
// +explorerExample{xpathBinding,XPath Binding example} shows loading XML and processing it with
// XPaths.
// <P>
// <B>WSDL integration</B>
// <P>
// If you have a choice between WSDL and REST integration, we recommend REST integration - it's
// simpler, requires less specialized knowledge, is easier to troubleshoot and is faster.  If
// you need to use WSDL, see the +link{group:wsdlBinding,WSDL Binding Overview}.
// <P>
// <H4>Round Tripping: Loading, Editing and Saving</H4>
// <P>
// For WSDL web services, see the +link{group:wsdlBinding,WSDL binding topic} first.
// <P>
// When using RestDataSource, see the +link{RestDataSource} docs for message formats for
// saving, as well as expected responses.
// <P>
// When a user triggers a DSRequest (eg, completes an inline edit in a grid), the request
// data will be sent to the dataURL.  The +link{group:dataSourceOperations,DataSource protocol}
// describes request and response data expected for each operation type.  
// <P>
// By using settings such as +link{operationBinding.dataProtocol}, you can control how
// DSRequests are sent to your backend so that you can handle them most easily.  By using the
// same properties used to initially load data (eg
// +link{operationBinding.recordXPath,recordXPath}), you can control how SmartClient forms the
// DSResponses that are then interpreted by +link{DataBoundComponent,databound components}.
// <P>
// <b>Controlling how DSRequests are sent</b>
// <P>
// According to the +link{attr:operationBinding.dataProtocol,protocol} being used, the
// +link{group:dataSourceOperations,DataSource request data}, if any, either becomes HTTP
// params (sent by GET or POST), or an XML message as put together by
// +link{method:DataSource.xmlSerialize()}.  For a DataSource invoking a WSDL-described web
// service, XML serialization automatically handles namespacing and SOAP encoding.
// <P>
// Note that, by default, just +link{attr:dsRequest.data} is sent, not any of the metadata such
// as +link{attr:dsRequest.startRow}.  This can be customized via
// +link{method:DataSource.transformRequest()}.
// <P>
// The URL to contact is set via the +link{attr:operationBinding.dataURL,dataURL}
// property.  If using a Web Service, the <code>dataURL</code> defaults to the service location
// URL embedded in the WSDL file.
// <P>
// For example, in the default configuration for non-WSDL binding, since
// +link{operationBinding.dataProtocol,dataProtocol} is "getParams", +link{dsRequest.data} is
// sent as HTTP params in an HTTP "GET" operation.  Given:
// <ul>
// <li> changes to an existing record, hence an "update" request
// <li> a +link{dataSourceField.primaryKey,primaryKey} field of "id" with value "5" on the
// record to be updated
// <li> a field "age" being changed to "32"
// <li> "dataURL" of "save.php"
// </ul>
// You will see an HTTP GET to the URL <code>save.php?id=5&age=32</code>.
// <P>
// <b>Forming a DSResponse from the response data</b>
// <P>
// A +link{class:DSResponse} is created from the response data by using XPath
// expressions declared in the schema (+link{attr:operationBinding.recordXPath,recordXPath} and 
// +link{attr:DataSourceField.valueXPath,valueXPath}) to extract DataSource record and field
// values.
// <P>
// See the +explorerExample{xmlEditSave,"Edit and Save"} example for sample XML responses for
// all four operationTypes.  
// <P>
// Similar to input processing, by default DataSource layer metadata, such as
// +link{attr:dsResponse.startRow}, is not extracted from the response data.  You can
// implement +link{method:DataSource.transformResponse()} to fill out the metadata fields of
// the +link{class:DSResponse}, in order to allow more DataSource features, such as paging and
// validation errors, to be used with a web service that supports such features.  
// <P>
// See the +explorerExample{xmlServerValidationErrors, XML} and 
// +explorerExample{jsonServerValidationErrors, JSON} versions of the transformResponse()
// example for an example of providing validation errors in XML or JSON responses.
//
// @visibility xmlBinding
// @treeLocation Concepts/Client-Server Integration
// @treeLocation Client Reference/Data Binding
// @title Client-side Data Integration
//<

//> @groupDef serverScript
// SmartClient allows you to embed "scriptlets" directly in your .ds.xml file to take care of
// simple business logic without having to create a separate file or class to hold the logic.
// <P>
// These scriptlets can be written in any language supported by the Java "JSR 223" standard,
// including Java itself, as well as languages such as Groovy, JavaScript, Velocity, Python, Ruby,
// Scala and Clojure.
// <P>
// Scriptlets are automatically recompiled when you change the .ds.xml file - just reload the page
// and the SmartClient Server Framework automatically notices the modified DataSource file and
// uses the new scriptlets.
// <P>
// <b>Declaring Scriptlet Language</b>
// <P>
// You can set the default system-wide language in +link{server_properties,server.properties}
// by setting
// <code>script.defaultLanguage</code>:
// <pre>
//     script.defaultLangauge: "groovy"
// </pre>
// Alternatively, anywhere a scriptlet is allowed, you can use the "language" attribute to declare
// the language.  For example:
// <pre>
//    &lt;operationBindings&gt;
//       &lt;operationBinding operationType="add"&gt;
//           &lt;script language="groovy"&gt;
//              ... Groovy code ...
//           &lt;/script&gt;
//       &lt;/operationBinding&gt;
//    &lt;/operationBindings&gt;
// </pre>
// <h3>Error Reporting</h3>
// <P>
// If your scriptlet crashes, this is reported in the server-side log along with the line number
// of the crash.  
// <pre>
//      &lt;!-- crash will be reported at line 1 --&gt;
//      &lt;script&gt;crash()&lt;/script&gt;
//
//      &lt;!-- crash will be reported at line 2 --&gt;
//      &lt;script&gt;
//          crash()
//      &lt;/script&gt;
// </pre>
// It's common practice to use a CDATA tag so that XML-special characters such as &lt; do not have to be
// quoted as &amp;lt;.  When doing this, be aware that the line numbering will still start <b>from the
// &lt;script&gt; tag</b>, not the CDATA tag.  For example:
// <pre>
//      &lt;!-- crash will be reported at line 3 --&gt;
//      &lt;script&gt;
//          &lt;![CDATA[ 
//             5 < crash()
//          ]]&gt;
//      &lt;/script&gt;
//
//      &lt;!-- crash will be reported at line 2 --&gt;
//      &lt;script&gt;&lt;![CDATA[ 
//           5 < crash()
//      ]]&gt;&lt;/script&gt;
// </pre>
// <P>
// <h3>Java Imports</h3>
// <P>
// You can import Java libraries by placing a &lt;scriptImport&gt; tag immediately before a
// &lt;script&gt; or &lt;serverCondition&gt; tag, like so:
// <pre>
//     &lt;scriptImport&gt;javax.servlet.http.*&lt;/scriptImport&gt;
//     &lt;script language="groovy"&gt;
//         String sessionId = session.getId();
//         ...
// </pre>
// There is also a system-wide set of default imports:
// <pre>
// java.util.*
// javax.servlet.http.*
// com.isomorphic.base.Config
// com.isomorphic.util.*
// com.isomorphic.datasource.*
// com.isomorphic.rpc.RPCManager
// </pre>
// You can override these in +link{server_properties,server.properties} via the property
// <code>script.defaultImports</code>, which takes a comma- or space-separated list of packages or
// classes (like the above).
// <P>
// Dynamic languages such as Groovy or JavaScript allow you to place an import inside the script
// itself as well.
// <P>
// <h3>Available Languages</h3>
// <P>
// The Oracle JDK and JRE include support for JavaScript scripting via the Rhino engine.
// Alternatively, you may wish to consider +externalLink{http://code.google.com/p/jav8/,jav8},
// which is a Javascript JSR223 scripting implementation based on the V8 engine, giving 
// Javascript execution speeds comparable to Google Chrome.
// <p>
// For convenience, SmartClient also bundles a .jar providing Groovy support from 
// +externalLink{http://www.groovy-lang.org/}, which uses the Apache license.  We also include
// a .jar file providing Java language support (however see below for limitations).  This
// implementation is based on the BSD-licensed
// +externalLink{http://java.net/projects/scripting/, Java.net} implementation, but enhanced by
// Isomorphic to work around container-specific classloader issues that arise when running Java
// language scripting inside a servlet container and trying to reference common objects of the
// servlet API itself.  See +link{sunNotice} for licensing information.
// <P>
// There are <b>many</b> other languages available, sometimes with multiple implementations, and
// they are best found via web search.
// <P>
// <h3>Standard Headers &amp; Footers</h3>
// <P>
// You can define system-wide headers and footers for each language - code that is added before
// and after scriptlets wherever it is defined, and can set up variables or functions you use
// often.  To define the location of header and footer files, set
// <code>script.<i>languageName</i>.header</code> and <code>script.<i>languageName</i>.footer</code>
// in server.properties.  For example, these settings:
// <pre>
//   script.groovy.header: $webRoot/shared/header.groovy
//   script.groovy.footer: $webRoot/shared/footer.groovy
// </pre>
// would add the Groovy fragments found in header.groovy and footer.groovy to beginning and end of every
// scriptlet that declares language="groovy" (or declares no language if the default engine is
// "groovy").
// <P>
// <b>NOTE</b>: most scripting engines are available under several language names.  For example,
// the Rhino JavaScript engine registers both "javascript" and "ecmascript" as well as a few
// variations on letter case.  When using the "language" attribute on script tags, the exact value
// supplied is used to look up header and footer files via server.properties.  This means a
// language setting of "javascript" will find different header and footer files from a language of
// "JavaScript" even though both will execute via Rhino.
// <P>
// <h3>Java scriptlets and the default script wrapper</h3>
// <P>
// Although it's not usually considered a "scripting language", using the Java language for
// scriplets has the advantage that developers do not need to understand two languages in order to
// modify server-side code.  However, using Java for scripting presents special challenges,
// because unlike true scripting languages, in Java a piece of code cannot be compiled unless it
// forms a valid class definition.  
// <P>
// For this reason, by default every Java scriplet has an implicit wrapper added around it which
// makes it into a class definition of a trivial class with one method, and your scriptlet code
// forms the body of that method, after a series of local variables have been set up to allow
// convenient access to context variables.  The header and footer files you've defined, if any,
// appear before and after your scriptlet, still within the method body.
// <P>
// This makes Java viable as a scripting language despite its verbosity - if the actual business
// logic to be executed consists of just a few lines of Java, your overall scriptlet will be only
// that long instead of being forced to contain a complete class definition.
// <P>
// The automatic wrapping of Java code can be disabled by setting
// <code>script.java.useDefaultScriptWrapper</code> to false in server.properties.  In this case
// any scriptlet must contain a valid class definition like the below - context variables need to
// be manually retrieved from the ScriptContext object instead of being automatically
// available as local variables, and the attribute "evalResult" is used to return data in lieu of
// using a <code>return</code> statement.
// <P>
// <pre>
// class Temp {
//     private static ScriptContext ctx;
//     public static void setScriptContext(ScriptContext context) {
//         ctx = context;
//     }
//     public static void main(String[] args) {
//         String result = "Hello World!";
//         ctx.setAttribute("evalResult", result, 
//                          ScriptContext.ENGINE_SCOPE);
//     }
// }
// </pre>
// All scriptlets must also import javax.script.ScriptContext.  For obvious reasons setting
// <code>useDefaultScriptWrapper</code> to false is not recommended.
// <P>
// <h3>Returning values and JavaScript</h3>
// <P>
// Scriptlets written in Java <b>must</b> use a <code>return</code> statement to return a
// result from the scriptlet.  Scriptlets written in JavaScript <code>must not</code> use a
// <code>return</code> as Rhino will report this as an error - the JavaScript code is not
// executed in the scope of a function, and only functions can <code>return</code>.
// <P>
// Instead, JavaScript scriptlets should simply end with a statement indicating the value
// they would like to return.  For example:
// <pre>
//     // if used as the last line, the scriptlet 
//     // returns the result of dsRequest.execute();
//     dsRequest.execute();
//
//     // if you already have the value as a variable,
//     // just end with the variable name plus semicolon
//     var dsResponse = dsRequest.execute();
//     dsResponse;
//
//     // add a line like this to force returning null 
//     // instead of the result of the previous line of code
//     null;
// </pre>
// Groovy makes the <code>return</code> statement optional, and like JavaScript, will take the
// value of the last statement as the returned value if there is no explicit
// <code>return</code>.
// <P>
// Other languages supported by JSR223 may have other special semantics for returning data - see
// their documentation for details.
// <P>
// <h3>Available context variables for scriptlets</h3>
// <P>
// Context variables that are available to a scriptlet are explained in the documentation for the
// particular property where a scriptlet may be declared, for example,
// +link{operationBinding.script} and +link{validator.serverCondition}.
// <P>
// In most JSR223 languages, context variables are available as ordinary local variables and you
// can simply refer to them directly in your scriptlet.  This includes Java, so long as
// useDefaultScriptWrapper is left in it's default setting (see above).
//
// @title Server Scripting
// @treeLocation Concepts
// @example scriptingUserSpecificData
// @example scriptingValidation
// @visibility external
//<

//> @groupDef sunNotice
// The support provided in SmartClient for the use of Java as a server-side scripting language
// is based on Sun's "java-engine" implementation.  The license for that software requires 
// that we reproduce the copyright notice and disclaimer in our documentation:
// <pre>
//  Copyright (C) 2006 Sun Microsystems, Inc. All rights reserved. 
//  Use is subject to license terms.
// 
//  Redistribution and use in source and binary forms, with or without modification, are 
//  permitted provided that the following conditions are met: Redistributions of source code 
//  must retain the above copyright notice, this list of conditions and the following disclaimer.
//  Redistributions in binary form must reproduce the above copyright notice, this list of 
//  conditions and the following disclaimer in the documentation and/or other materials 
//  provided with the distribution. Neither the name of the Sun Microsystems nor the names of 
//  is contributors may be used to endorse or promote products derived from this software 
//  without specific prior written permission. 
//  
//  THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS
//  OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY 
//  AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER 
//  OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR 
//  CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR 
//  SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON 
//  ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE
//  OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
//  POSSIBILITY OF SUCH DAMAGE.
// </pre>
//
// @title Sun's java-engine implementation - Notice and Disclaimer
// @treeLocation Concepts
// @visibility external
//<

//> @groupDef nonJavaBackend
// While SmartClient's extensive server features are lost if you cannot install the Java-based
// server framework, SmartClient UI components can be integrated with any server technology.
// This topic provides pointers to documentation that is most relevant for this type of
// integration.
// <p>
// <var class="smartgwt">
// If you are using a non-Java backend such as PHP or .NET, you may want to consider 
// SmartGWT's sister product &#83;martClient, which has the same functionality but allows
// programming in JavaScript instead of Java.  However if you prefer to use Java on the client,
// this is fine - SmartGWT and SmartClient have identical capabilities for integrating with
// non-Java servers.
// <p>
// When run on a SmartGWT project, the GWT compiler produces ordinary web assets - JavaScript
// files, .html files, images, etc - which can be deployed on any web server like any other web
// assets.  How best to integrate GWT into your development workflow is outside the scope of
// this documentation, however, consider using the SmartGWT server-side HttpProxy (mentioned
// below) as a means of using a Java IDE to develop GWT Java code in the usual way, while
// allowing your GWT application to contact services hosted on another server (which is
// running your PHP, .NET or other server code).  The HttpProxy can then be eliminated from
// your final deployment by deploying the web assets produced by GWT directly on the server
// running your non-Java server code.
// </var>
// <var class="smartclient">
// <h4>Installation</h4>
// <P>
// As described in +link{group:iscInstall, <i>Deploying SmartClient</i>}, for a client-only
// integration, installation consists of just copying a directory of JavaScript and media files
// to your webserver.
// <P>
// <h4>Creating Components</h4>
// <P>
// SmartClient components can be included in any .html page, including dynamically generated pages
// produced by .php or .asp files.  The SmartClient libraries can be included in the page as
// follows:
// <pre>
// &lt;HTML&gt;&lt;HEAD&gt;
// &lt;SCRIPT&gt;var isomorphicDir="../isomorphic/";&lt;/SCRIPT&gt;
// &lt;SCRIPT SRC=../isomorphic/system/modules/ISC_Core.js&gt;&lt;/SCRIPT&gt;
// &lt;SCRIPT SRC=../isomorphic/system/modules/ISC_Foundation.js&gt;&lt;/SCRIPT&gt;
// &lt;SCRIPT SRC=../isomorphic/system/modules/ISC_Containers.js&gt;&lt;/SCRIPT&gt;
// &lt;SCRIPT SRC=../isomorphic/system/modules/ISC_Grids.js&gt;&lt;/SCRIPT&gt;
// &lt;SCRIPT SRC=../isomorphic/system/modules/ISC_Forms.js&gt;&lt;/SCRIPT&gt;
// &lt;SCRIPT SRC=../isomorphic/system/modules/ISC_DataBinding.js&gt;&lt;/SCRIPT&gt;
// &lt;SCRIPT SRC=../isomorphic/skins/SmartClient/load_skin.js&gt;&lt;/SCRIPT&gt;
// &lt;/HEAD&gt;&lt;BODY&gt;
// ...</pre>
// SmartClient components can then be created via normal JavaScript:
// <pre>
// &lt;SCRIPT&gt;
// isc.Button.create({
//     title:"Button",
//     click:"isc.say('Hello World')"
// });
// &lt;/SCRIPT&gt;</pre>
// This approach is discussed in more detail in the 
// +externalLink{/docs/SmartClient_Quick_Start_Guide.pdf,QuickStart Guide}, Chapter 4, <i>Coding</i>.
// Note that JavaScript-based component instantiation is currently the recommended approach, and
// most examples are provided in the JavaScript format.
// </var>
// <P>
// <h4>Data Loading / Data Binding</h4>
// <P>
// The primary focus of SmartClient integration is connecting DataSource operations to your
// server.  The +link{group:clientDataIntegration,Client-side Data Integration} chapter covers
// the key approaches, including cookbook approaches for REST-based integration with any server
// that can return XML or JSON over HTTP.
// <P>
// <h4>Simple RPCs (non-DataSource requests)</h4>
// <P>
// You can implement simple RPCs by having your server output 
// +externalLink{http://www.json.org/,JSON} (JavaScript Object Notation) and using 
// +link{RPCRequest.evalResult} to directly turn JSON results into live JavaScript objects.
// +link{RPCRequest.serverOutputAsString} lets you load arbitrary server results, including
// JSON results that need to be processed before they can be eval()'d.
// <P>
// Alternatively, if you are familiar with WSDL web services, you can implement simple RPCs as
// web service operations: use +link{XMLTools.loadWSDL()} to load the service definition, and
// then use +link{WebService.callOperation()} to call the operations.  We don't generally
// recommend this approach unless you are already deeply familiar with WSDL - it's far more
// complicated that producing and consuming JSON.
// <P>
// <h4>HTTPProxy: Cross-site or cross-port data loading</h4>
// <P>
// If you develop a prototype using the SmartClient SDK and SmartClient Java Server, and then
// you migrate the prototype to another server technology, you need to be aware that the
// SmartClient Java Server includes an HTTPProxy servlet that allows SmartClient interfaces to
// contact servers other than the origin server (bypassing what is called the
// +externalLink{http://www.google.com/search?q=same+origin+policy,"same origin policy"}).
// <P>
// SmartClient uses the HttpProxy automatically when needed, so it may not be obvious that the
// HTTPProxy is in use.  Then, your migrated application will encounter errors attempting to
// contact the HTTPProxy servlet.
// <P>
// To avoid these errors, ensure that all services that your application uses are accessed
// using the same hostname and port as the page was loaded from.  In particular, watch for WSDL
// files, which contain the service URL - you may need to use +link{webService.setLocation()}
// to ensure that the web service URL and page URL match.
// <P>
// If your production application really does need to access services or content hosted on
// other servers, typical practice is to pursue normal SmartClient integration with your
// server, then write server-side code that contacts other hosts on behalf of your SmartClient
// interface.
//
// @visibility xmlBinding
// @treeLocation Concepts
// @title .NET, PHP, Serverless Integration
//<

// JSP tag docs
// ---------------------------------------------------------------------------------------

//> @groupDef jspTags
// 
// The SmartClient Java Server component ships with a number of custom JSP tags designed to
// make development with SmartClient easier.  The custom tags are defined in
// <code>[webroot]/WEB-INF/iscTaglib.xml</code> in the SDK package.  To make use of these tags, make
// sure you have the following entry in your web.xml (the SDK already contains this entry by
// default):<br>
// <pre>
// &lt;taglib&gt;
//     &lt;taglib-uri&gt;isomorphic&lt;/taglib-uri&gt; 
//     &lt;taglib-location&gt;/WEB-INF/iscTaglib.xml&lt;/taglib-location&gt; 
// &lt;/taglib&gt;
// </pre>
// <p>
// Also, the JSP page making use of the tag must load the custom taglib via the following
// directive:<br>
// <pre>
// &lt;%@ taglib uri="/WEB-INF/iscTaglib.xml" prefix="isomorphic" %&gt;
// </pre>
// All SmartClient JSP tags produce either HTML or JavaScript output, so you can easily see
// what any given tag is generating by doing a "View->Source" in your browser after browsing to
// the JSP that contains your tag.  Tags that produce HTML must be located in the HTML BODY
// context in your JSP - that is, outside of any <code>&lt;SCRIPT&gt;</code> tags and inside
// <code>&lt;BODY&gt;</code> tags.  Tags that produce JavaScript must be located inside
// <code>&lt;SCRIPT&gt;</code> tags.
//
// @visibility external
// @requiresModules SCServer    
// @title SmartClient JSP Tags
//<

//> @groupDef loadISCTag
//
// <i>produces:</i> HTML
// <p>
// This tag sets window.isomorphicDir and loads a default set of SmartClient modules and a
// skin.  SmartClient modules are JavaScript files that need to be loaded via 
// <code>&lt;SCRIPT SRC=/some/filename.js&gt;&lt;/SCRIPT&gt;</code>
// blocks as part of the loading HTML page.  The modules themselves are stored in the
// <code>[webroot]/isomorphic/system/modules</code> directory.  A skin defines the look and feel of your
// website and typically consists of at least one CSS file and some SmartClient UI Component
// default overrides.  Skins are located in <code>[webroot]/isomorphic/skins</code>.  Loading a skin
// consists of loading the corresponding <code>load_skin.js</code> - residing at top-level in
// each skin directory.
// <p>
// The default set of modules loaded by this tag are: <code>Core, Foundation, Containers,
// Grids, Forms, DataBinding</code>.  The default skin is the <code>SmartClient</code> skin.
// So, for example inserting this into your JSP:
// <pre>
// &lt;isomorphic:loadISC/&gt;
// </pre>
// Produces the following output:
// <pre>
// &lt;SCRIPT&gt;window.isomorphicDir='isomorphic/';&lt;/SCRIPT&gt;
// &lt;SCRIPT SRC=isomorphic/system/modules/ISC_Core.js&gt;&lt;/SCRIPT&gt;
// &lt;SCRIPT SRC=isomorphic/system/modules/ISC_Foundation.js&gt;&lt;/SCRIPT&gt;
// &lt;SCRIPT SRC=isomorphic/system/modules/ISC_Containers.js&gt;&lt;/SCRIPT&gt;
// &lt;SCRIPT SRC=isomorphic/system/modules/ISC_Grids.js&gt;&lt;/SCRIPT&gt;
// &lt;SCRIPT SRC=isomorphic/system/modules/ISC_Forms.js&gt;&lt;/SCRIPT&gt;
// &lt;SCRIPT SRC=isomorphic/system/modules/ISC_DataBinding.js&gt;&lt;/SCRIPT&gt;
// &lt;SCRIPT src=isomorphic/skins/SmartClient/load_skin.js&gt;&lt;/SCRIPT&gt;
// </pre>
// <b><u>Tag Attributes:</u></b>
// <p>
// <b>skin</b><br>
// <i>value format</i>: Name of skin to load.  See the <code>[webroot]/isomorphic/skins</code> directory
// for a list of available skins.  You can also create your own skin by copying an existing
// skin directory recursively, modifying anything you need, and using your new skin name as the
// value here.<br>
// <i>default value</i>: "SmartClient"
// <p>
// This selects a skin other than the default.  The value of this attribute is used as a path
// component to <code>load_skin.js</code>.  For example, specifying:
// <pre>
// &lt;isomorphic:loadISC skin="standard"/&gt;
// </pre>
// Produces the following <code>load_skin.js</code> directive (in addition to all the standard
// module loads):
// <pre>
// &lt;SCRIPT src=isomorphic/skins/standard/load_skin.js&gt;&lt;/SCRIPT&gt;
// </pre>
// <p>
// <b>skinDir</b><br>
// <i>value format</i>: path or URL up to and including the name of the skin<br>
// <i>default value</i>: automatically derived from the value of the <code>skin</code>
// attribute
// <p>
// This attribute is usable in lieu of the <code>skin</code> attribute and allows you to
// specify the full path or URL to the directory containing the load_skin.js for your skin.
// This allows you to move the directory containing the skin anywhere you want - including to a
// completely different server.
// <p>
// <b>includeModules</b><br>
// <i>value format</i>: Comma separated list of additional modules to load.  See the
// <code>[webroot]/isomorphic/system/modules</code> directory for a list of loadable modules.  The
// leading "ISC_" is optional, spaces after commas are also optional.<br>
// <i>default value</i>: NONE
// <p>
// This attribute allows you to specify SmartClient modules that you want to load in addition
// to the default set of modules. For example, specifying:
// <pre>
// &lt;isomorphic:loadISC includeModules="Analytics"/&gt;
// </pre>
// Produces the following output:
// <pre>
// &lt;SCRIPT&gt;window.isomorphicDir='isomorphic/';&lt;/SCRIPT&gt;
// &lt;SCRIPT SRC=isomorphic/system/development/ISC_Core.js&gt;&lt;/SCRIPT&gt;
// &lt;SCRIPT SRC=isomorphic/system/development/ISC_Foundation.js&gt;&lt;/SCRIPT&gt;
// &lt;SCRIPT SRC=isomorphic/system/development/ISC_Containers.js&gt;&lt;/SCRIPT&gt;
// &lt;SCRIPT SRC=isomorphic/system/development/ISC_Grids.js&gt;&lt;/SCRIPT&gt;
// &lt;SCRIPT SRC=isomorphic/system/development/ISC_Forms.js&gt;&lt;/SCRIPT&gt;
// &lt;SCRIPT SRC=isomorphic/system/development/ISC_DataBinding.js&gt;&lt;/SCRIPT&gt;
// &lt;SCRIPT SRC=isomorphic/system/development/ISC_Analytics.js&gt;&lt;/SCRIPT&gt;
// &lt;SCRIPT src=isomorphic/skins/SmartClient/load_skin.js&gt;&lt;/SCRIPT&gt;
// </pre>
// Notice the addition of the line that loads <code>ISC_Analytics.js</code>.  SmartClient
// modules must be loaded in a particular order and may have dependencies on other modules.
// The <code>loadISC</code> automatically resolves any missing dependencies and includes them
// in the list of modules to load.  The list of modules to load is also automatically reordered
// as required by this tag, so you can specify just the modules you care about, in any order,
// and <code>loadISC</code> will do the right thing.
// <p>
// <b>modules</b><br>
// <i>value format</i>: Comma separated list of modules to load.  See the
// <code>[webroot]/isomorphic/system/modules</code> directory for a list of loadable modules.  The
// leading "ISC_" is optional, spaces after commas are also optional.<br>
// <i>default value</i>: "Core, Foundation, Containers, Grids, Forms, DataBinding"
// <p>
// Specifying this attribute overrides the default set of modules loaded by
// <code>loadISC</code>. This is useful if you don't need to load all of the default modules on
// a particular page.
// <p>
// <b>isomorphicURI</b><br>
// <i>value format</i>: absolute or relative (from current URI) path to the
// <code>isomorphic</code> directory (by default, located in webRoot).<br>
// <i>default value</i>: automatically derived
// <p>
// This value sets <code>window.isomorphicDir</code> and defines the base path to SmartClient
// modules from the currently executing JSP.  By default, this value is automatically derived
// and you don't need to override it.  Generally, an override is only required if you're
// pointing multiple web application deployments at one base URL for cache coherence reasons.
// <p>
// <b>defer</b><br>
// <i>value format</i>: boolean - acceptable values: "true" or "false"<br>
// <i>default value</i>: "false"
// <p>
// This attribute applies only if you're using the optional Network Performance component of
// SmartClient.  By default, the modules specified by the <code>loadISC</code> tag are loaded
// synchronously - that is, the browser waits for each module to download before rendering
// any HTML on the page.  Specifying <code>defer="true"</code> makes the module load
// asynchronous.  In this mode, all the page content except the modules specified by
// <code>loadISC</code> is loaded and rendered first, and then the SmartClient modules are
// loaded.  The asynchronous mode makes it possible to render some content and allow the user
// to interact with the site immediately, while SmartClient modules load in the background.
// <p>
// Note that typically SmartClient modules are delivered compressed with cache headers, so the
// download delay really only affects first time users of your site, those whose cached
// versions have expired, or if you've upgraded your SmartClient module to a newer version that
// the user has cached.
// <p>
// Using this directive is equivalent to calling +link{FileLoader.loadISC()}.
// <p>
// <b>cacheOnly</b><br>
// <i>value format</i>: boolean - acceptable values: "true" or "false"<br>
// <i>default value</i>: "false"
// <p>
// This attribute applies only if you're using the optional Network Performance component of
// SmartClient.  This works like <code>defer</code> (see above), except that the modules are
// only loaded into the browser cache, but not executed.  This means that when the specified
// modules download, they'll be added to the browser cache, but will not be available to the
// execution context of the page.  Actually executing the module code takes some processing
// time on the client, whereas simply caching it does not.  The typical use case for this
// directive is on some intermediary page before the user gets to a SmartClient-enabled page.
// For example, you could include a cacheOnly <code>loadISC</code> tag on a login page that
// does not use SmartClient components, but protects a SmartClient-enabled application.  While
// the user is typing in their credentials, the SmartClient modules will cache in the
// background, then once the user logs in and is sent to a SmartClient-enabled page, the
// modules are simply read from cache.
// <p>
// Using this directive is equivalent to calling +link{FileLoader.cacheISC()}.
// <p>
// <b>onload</b><br>
// <i>value format</i>: String - any javascript code<br>
// <i>default value</i>: NONE
// <p>
// This attribute applies only if you're using the optional Network Performance component of
// SmartClient.  This attribute is usable in conjunction with <code>defer</code> and
// <code>cacheOnly</code> attributes.  This attribute allows you to specify a callback to
// execute when loading or caching of the specified modules has completed.  You can use this
// to, for example, replace portions of the page with SmartClient components once SmartClient
// has loaded.
// <p>
// The value of this attribute is passed as the <code>onload</code> handler to the generated
// +link{FileLoader.cacheISC()} or +link{FileLoader.loadISC()} call as appropriate.
// <p>
// <b>useSimpleNames</b><br>
// <i>value format</i>: boolean - acceptable values: "true" or "false"<br>
// <i>default value</i>: "true"
// <p>
// Setting this attribute to false puts SmartClient in namespace mode.  See +link{class:isc}
// for more info.
//
// @visibility external
// @requiresModules SCServer    
// @treeLocation Java Server Reference/SmartClient JSP Tags
// @title &lt;isomorphic:loadISC&gt;
//<


//> @groupDef loadModulesTag
//
// <i>produces:</i> HTML
// <p>
// This tag works just like +link{group:loadISCTag} except it does not load a skin.  All other
// attributes are supported on this tag just as on <code>loadISC</code>.  This tag is useful if
// you have a single "header" JSP that has the <code>loadISC</code> tag that you then include in other
// SmartClient-enabled JSPs that require additional modules.  The JSPs that require additional
// modules can then use the <code>loadModules</code> to load additional SmartClient modules.
//
// @visibility external
// @requiresModules SCServer    
// @treeLocation Java Server Reference/SmartClient JSP Tags
// @title &lt;isomorphic:loadModules&gt;
//<


//> @groupDef loadDSTag
//
// <i>produces:</i> JavaScript
// <p>
// This tag converts a SmartClient DataSource or SimpleType defined in XML to JavaScript for
// use in databinding on the client (browser).
// <p>
// Note that this JSP tag must be surrounded by &lt;SCRIPT&gt; tags in the JSP because it
// generates JavaScript code.  Like other tags that generate JavaScript code, this tag can be
// used in a JSP that is included from your main page in order to create separate
// cacheability.  For example:<pre>
//     &lt;SCRIPT SRC="myDataSources.jsp"&gt;&lt;/SCRIPT&gt;
// </pre>
// <p>
// <b><u>Tag Attributes:</u></b>
// <p>
// <b>ID</b><br>
// <i>value format</i>: String - ID of datasource or simpleType to load<br>
// <i>default value</i>: NONE
// <p>
// This attribute specifies the name of the dataSource or simpleType that you wish to load.
// DataSources are located in <code>[webroot]/shared/ds</code> by default.  This location is
// changeable in <code>[webroot]/WEB-INF/classes/server.properties</code> by setting the config
// parameter <code>project.datasources</code> to the directory where your dataSources are
// located.
// We recommend that for prototyping, at least, you use the default directory.
// <p>
// For example:
// <pre>
// &lt;isomorphic:loadDS ID="supplyItem"/&gt;
// </pre>
// Would load the <code>supplyItem</code> DataSource.
// <p>
// You can also load multiple dataSources in one go by specifying a comma-separated list of 
// dataSource names as the ID.  For example:
// <pre>
// &lt;isomorphic:loadDS ID="supplyItem, employees, worldDS"/&gt;
// </pre>    
// See +link{group:dataSourceDeclaration} for more details on creating DataSources and an
// example.
// <p>
// See +link{class:SimpleType} for more details on how to define server-side SimpleType in
// xml format.
// <p>
// <b>name</b><br>
// <i>value format</i>: String - ID of datasource to load<br>
// <i>default value</i>: NONE
// <p>
// This is a synonym for the <code>ID</code> attribute.
// 
//
// @see group:dataSourceDeclaration 
//
// @visibility external
// @requiresModules SCServer    
// @treeLocation Java Server Reference/SmartClient JSP Tags
// @title &lt;isomorphic:loadDS&gt;
//<


//> @groupDef loadUITag
//
// <i>produces:</i> JavaScript
// <p>
// This tag converts SmartClient UI components declaratively specified in an XML "UI" file to
// JavaScript for execution in the browser.
// <p>
// The XML->JS translation works just like with +link{group:xmlTag}, except the XML is read
// from an external file instead of from the body of the tag.
// <p>
// Note that this JSP tag must be surrounded by &lt;SCRIPT&gt; tags in the JSP because it
// generates JavaScript code.  Like other tags that generate JavaScript code, this tag can be
// used in a JSP that is included from your main page in order to create separate
// cacheability.  For example:<pre>
//     &lt;SCRIPT SRC="myUIDefinitions.jsp"&gt;&lt;/SCRIPT&gt;
// </pre>
// <p>
// <p>
// <b><u>Tag Attributes:</u></b>
// <p>
// <b>name</b><br>
// <i>value format</i>: String - name of UI file to load (minus extension)<br>
// <i>default value</i>: NONE
// <p>
// This attribute specifies the name of the file that contains the UI components to translate.
// UI files are located in <code>[webroot]/shared/ui</code> by default.  This location is
// changeable in <code>[webroot]/WEB-INF/classes/server.properties</code> by setting the config
// parameter <code>project.ui</code> to the directory where your UI files are located.   
// We recommend that for prototyping, at least, you use the default directory.
// <p>
// For example:
// <pre>
// &lt;isomorphic:loadUI name="test"/&gt;
// </pre>
// Would translate declarative XML in the file <code>[webroot]/shared/ui/test.ui.xml</code> to
// JavaScript and output the results into the JSP output stream at the location of the tag.
//
// @see group:xmlTag
//
// @visibility external
// @requiresModules SCServer    
// @treeLocation Java Server Reference/SmartClient JSP Tags
// @title &lt;isomorphic:loadUI&gt;
//<

//> @groupDef xmlTag
//
// <i>produces:</i> JavaScript
// <p>
// This tag converts SmartClient UI components declaratively specified in the body of the tag to
// JavaScript for execution in the browser.
// <p>
// The XML->JS translation works just like with +link{group:loadUITag}, but the XML is read
// from the body of the tag.  If you wish, you can also specify an external filename, and XML
// will be read from that file, in addition to any XML encountered in the body of the tag.  If
// you do specify that an external file should be read, it is read from a path starting in 
// your webroot (as opposed to the <code>&lt;loadUI&gt;</code> tag, which looks in 
// <code>shared/ui</code>) 
// 
// <p>
// Note that this JSP tag must be surrounded by &lt;SCRIPT&gt; tags in the JSP because it
// generates JavaScript code.  Like other tags that generate JavaScript code, this tag can be
// used in a JSP that is included from your main page in order to create separate
// cacheability.  For example:<pre>
//     &lt;SCRIPT SRC="myUIDefinitions.jsp"&gt;&lt;/SCRIPT&gt;
// </pre>
// <p>
// Example of using this tag :
// <pre>
// &lt;isomorphic:XML&gt;
// &lt;Canvas&nbsp;backgroundColor="black"/&gt;
// &lt;/isomorphic:XML&gt;
// </pre>
// Would output the following JavaScript code:
// <pre>
// Canvas.create({
//   backgroundColor: "black"
// });
// </pre>
// <p>
// <b><u>Tag Attributes:</u></b>
// <p>
// <b>filename</b><br>
// <i>value format</i>: String - name of XML file to load (including the "XML" extension)<br>
// <i>default value</i>: NONE
// <p>
// This optional attribute specifies the name of an XML file to read and convert.
// <p>
// An example that specifies both a filename and some XML in the tag body:
// <pre>
// &lt;isomorphic:XML filename="test.xml"&gt;
//   &lt;Canvas backgroundColor="red"/&gt;
// &lt;/isomorphic:XML&gt;
// </pre>
//
// @see loadUITag
//
// @visibility external
// @requiresModules SCServer    
// @treeLocation Java Server Reference/SmartClient JSP Tags
// @title &lt;isomorphic:XML&gt;
//<

//> @groupDef loadXMLSchemaTag
//
// <i>produces:</i> JavaScript
// <p>
// Load an XML Schema (.xsd) file and create a +link{SchemaSet} object representing the loaded
// definitions. This tag works just like +link{XMLTools.loadXMLSchema()}, except it's
// synchronous and the result is server-cacheable.
// <p>
// Note that this JSP tag must be surrounded by &lt;SCRIPT&gt; tags in the JSP because it
// generates JavaScript code.  Like other tags that generate JavaScript code, this tag can be
// used in a JSP that is included from your main page in order to create separate
// cacheability.  For example:<pre>
//     &lt;SCRIPT SRC="myXMLSchemaDefinitions.jsp"&gt;&lt;/SCRIPT&gt;
// </pre>
// <p>
// <b><u>Tag Attributes:</u></b>
// <p>
// <b>url</b><br>
// <i>value format</i>: URL or URI
// <i>default value</i>: NONE
// <p>
// This attribute specifies the URL or URI of the XML Schema file to fetch and translate.  This
// can be either a remote URL - e.g: <code>http://host:port/schemaFile.xsd</code> or a relative
// or absolute URI to a file local to this container - e.g: <code>/some/schemaFile.xsd</code>
// or <code>../some/schemaFile.xsd</code>.  If the url is a remote URL, then an HTTP request
// will be made for the file.  If it is local, it will be fetched from disk using standard
// Servlet APIs (<code>ServletContext.getResourceAsStream()</code>).
// <p>
// <b>cache</b><br>
// <i>value format</i>: Integer (number of seconds to cache result)
// <i>default value</i>: 3600 (1 hour)
// <p>
// This attribute specifies the number of seconds for which the fetched XML Schema is cacheable
// on the server.  Fetching an XML Schema file from a remote server can cause a significant
// delay in JSP processing, and XML Schema files rarely change outside of a development
// environment.  Set this value to zero to disable caching.
//
// @see XMLTools.loadXMLSchema()
//
// @visibility xmlBinding
// @requiresModules SCServer    
// @treeLocation Java Server Reference/SmartClient JSP Tags
// @title &lt;isomorphic:loadXMLSchema&gt;
//<

//> @groupDef loadWSDLTag
//
// <i>produces:</i> JavaScript
// <p>
// Load a WSDL file and create an instance of +link{WebService} that allows invoking operations
// and binding DataSources to web service operations. This tag works just like
// +link{XMLTools.loadWSDL()}, except it's synchronous and the result is server-cacheable.
// <p>
// Note that this JSP tag must be surrounded by &lt;SCRIPT&gt; tags in the JSP because it
// generates JavaScript code.  Like other tags that generate JavaScript code, this tag can be
// used in a JSP that is included from your main page in order to create separate
// cacheability.  For example:<pre>
//     &lt;SCRIPT SRC="myWebServiceDefinition.jsp"&gt;&lt;/SCRIPT&gt;
// </pre>
// <p>
// <b><u>Tag Attributes:</u></b>
// <p>
// <b>url</b><br>
// <i>value format</i>: URL or URI
// <i>default value</i>: NONE
// <p>
// This attribute specifies the URL or URI of the WSDL file to fetch and translate.  This can
// be either a remote URL - e.g: <code>http://host:port/wsdlFile.wsdl</code> or a relative or
// absolute URI to a file local to this container - e.g: <code>/some/wsdlFile.wsdl</code> or
// <code>../some/wsdlFile.wsdl</code>.  If the url is a remote URL, then an HTTP request will
// be made for the file.  If it is local, it will be fetched from disk using standard Servlet
// APIs (<code>ServletContext.getResourceAsStream()</code>).
// <p>
// <b>cache</b><br>
// <i>value format</i>: Integer (number of seconds to cache result)
// <i>default value</i>: 3600 (1 hour)
// <p>
// This attribute specifies the number of seconds for which the fetched WSDL is cacheable on
// the server.  Fetching a WSDL file from a remote server can cause a significant delay in JSP
// processing, and WSDL files rarely change outside of a development environment.  Set this
// value to zero to disable caching.
//
// @see group:wsdlBinding
// @see XMLTools.loadWSDL()
//
// @visibility xmlBinding
// @requiresModules SCServer    
// @treeLocation Java Server Reference/SmartClient JSP Tags
// @title &lt;isomorphic:loadWSDL&gt;
//<

//> @groupDef jsStringTag
//
// <i>produces:</i>JavaScript
// <p>
// This tag takes everything in its body and outputs a correctly-escaped JavaScript string.
// This is useful for capturing HTML for display in a Canvas or Label, for example.
// <p>
// <b><u>Tag Attributes:</u></b>
// <p>
// <b>var</b><br>
// <i>value format</i>: Any legal JavaScript variable name
// <i>default value</i>: NONE
// <p>
// If specified, the escaped string is assigned to a newly created variable of the specified
// name.  e.g: var foo = "bar";
// <p>
// <b>filename</b><br>
// <i>value format</i>: webRoot-relative path to file
// <i>default value</i>: NONE
// <p>
// If specified, the resulting string content is loaded from the specified file instead of
// from the tag body.
//
// @visibility external
// @requiresModules SCServer
// @treeLocation Java Server Reference/SmartClient JSP Tags
// @title &lt;isomorphic:jsString&gt;
//<



//> @groupDef xmlClientVsServer
// Server benefits
// - faster client-side processing
// Server neutral
// - heavy customization of XML transform, if any, written in Java
//
// Client benefits
// - faster server-side processing
// Client neutral
// - heavy customization of XML transform, if any, written in JavaScript
//<
