The significance of XML standards in e

Suitability of OAG (Open Applications Group) 's XML Spec. for BSPs

a few
subjective impressions
by Todd Boyle

OAG, the Open Applications Group, is at http://www.openapplications.org.

Business Services Providers (BSPs) are a segment of the ASP industry, who offer diverse functionality to subscribers over the internet. As described in an earlier article, the BSP industry is in great need of an effective, low-cost integration standard for interoperability right now.

The significance of standards in e-commerce can hardly be overstated. Standards are the only hope of avoiding the N-squared problem, in which every system has to maintain unique, customized connections with every other system.

OAG has achieved a worthwhile accomplishment, in defining business process abstractions and a common vocabulary for interoperation of different vendors' software. XML standards are useless without these semantics and business process assumptions. No standard, from EDI thru any of the XML dialects, have ever articulated such a far-reaching set of standard functionality. OAG has got integration implementations in production at more than 40 large enterprises.

The payoffs to companies in joining and participating in OAG and other major XML standards bodies are

immediate access to important components for their integration solutions,
sideways access, to components related to the products and schemas they adopt,
technical knowledge useful in determining criteria to select, buy or build complete solutions,
influence, in some cases, over other members' choices of solutions,
networking contacts,
sales leads,
recruiting leads,
leads for raising capital and so forth.

But the value proposition for the BSP or SME participant, is reduced by a number of factors.

direct costs of dues, days of work, travel expenses, etc.
loss of focus; longer time to market (if not managed carefully)
risk of hypnosis by groupthink or big vendor solutions

Standards bodies like OAG necessarily, have many members. Integrating the requirements of many constituents (each having many software requirements) necessarily adds long delays in achieving consensus. While it is true, an infinite number of members can be reached by a single broadcast, every proposal will trigger signals which change the spec, requiring another feedback loop. OAG is not unique in having many members of course. I have no evidence by which to judge its ineffective in compressing the feedback loops.

Because the members are predominantly very large companies, or very large ERP vendors serving those industries, there is a lack of urgency. There is a lack of both the corporate necessity to finish things quickly, and personal urgency by delegates to get things finished. This is also not unique to OAG, but rather, characteristic of standards processes involving large companies and mainframe and ERP vendors are involved.

OAG meetings work at a high level of abstraction, yet, the organization depends on these face to face meetings. SMEs cannot afford the cost of several days' meetings in faraway locations, over long-running abstract matters. The lack of urgency makes OAG inaccessible to SMEs or their consultants, particularly in the internet industry. The OAG membership fee of $10,000 also prohibits any significant small business participation in OAG.

The interests of SMEs of course are not represented at any standards bodies by their existing software vendors (Intuit, Sage, CA, etc.). Most SME software vendors have historically been dedicated to lock-in and eschewed open interfaces. Today's shrinkwrap accounting vendor wants to capture their customer base into an internet portal. They would hardly be found participating in anything like the OAG for small business, wishing instead that standards would never happen, or would be controlled by them.

There is also an absence of online discussion or deliberations in OAG. OAG operates by phone, email and meetings. Never mind that the OAG does not publish open discussions by web forum, newsgroup, or open mailing lists. Even within the private mailing list, there is no written discussion or deliberation! There are political and economic/ sales related factors in large organizations that apparently make these employees unlikely to participate in written format. The OAG would have to make a very determined effort to drive its dialog online or at least, a teleconferencing medium, to overcome this. For example,

conduct deliberations exclusively online, by removing other avenues of collaboration, or
invest in very high quality collaboration infrastructure that drives content online, or
stop publishing the completed standards, on a silver platter, and instead, publish only pointers to completed workgroup processes which have been conducted under criteria such as accessible online collaboration.
impose a burden on users of traversing a knowledge base and passing a quiz before they can download any standard

Obviously, any of these would impose a systemic cost that would slow the engines of OAG as presently constituted, for its primary EAI mission.

The pearl which draws SMEs and BSPs to OAGIS is the implied promise that by the size of its vocabulary and the structure of its BODs, there is a complete vocabulary, relatively free of name collisions, duplications, awkward structures, etc. Internet BSPs particularly need a well-planned XML vocabulary because they absolutely have insufficient time to work out all future business needs. It is impossible for them to predict all their future functions, or design XML that will mesh smoothly with those functions. Let's examine whether OAG fulfills that promise:

The perimeter of features encompassed by OAG is not bounded. The workgroup authoring the BODs have wide discretion in design choices. Because these are big-systems integration efforts the feature depth is, in fact, very deep compared with small business. This is not necessarily a bad thing for SMEs and in general it reduces the likelihood that OAGIS BODs will leave their needs unmet. However, the great feature depth makes the deliberation process more costly. It also adds too much time to the process. The Internet is here, now, and is going to be built one way or the other. If OAG is attractive it may be used as it stands. Otherwise it won't, and OAGIS will remain within its niche of high-end ERP systems.

Arbitrariness and omission of SME's needs: There are a disproportionate number of message BODs, and elements within those BODs, in certain areas like manufacturing. This imbalance creates biases on the definitions of the shared elements within Oagis_Segments and Oagis_Fields.

This is not necessarily a bad thing for small business as long as their tags can be found someplace in the OAG's feature set. However, that is most definitely not the case right now. Even the simplest SME accounting systems will have a substantial number of elements that are not represented by anything in the OAG's feature set.

What emerges is that there is arbitrariness in the OAG's selection of elements and attributes in its data structures. What you have is a set of BODs worked out by a particular set of vendors and customers. That set was not a small set but it was certainly not a large or diverse set of members compared to the entire SME community, even within the US.

And there is no systematic mechanism of sampling of external needs to make sure that elements included in the OAG vocabulary are statistically representative of typical midrange business or accounting installations, or well-balanced in relation to SMEs needs. This is no oversight: OAG process for new BODs is intentionally designed to be fairly rapid, and to favor action over inaction.

A tale of two General Ledgers: OMGs and OAGs

OMG: More participants from the finance and accounting. Less overhead from other parts of their companies. More freedom to be "Purists". The spec is closer to a universal common denominator. it has few elements that are not indisputably necessary or required for a General Ledger.

Here are some insights into the OMG's General ledger design:

typedef FbcCurrency::Money Money;
typedef sequence<wstring> wstringList; // sequence of type wstring
typedef sequence<boolean> booleanList; // sequence of type boolean
typedef wstring CurrencyMnemonic; // a reference to a single currency
typedef wstring TransactionId; // a single GL Transaction reference
typedef sequence<TransactionId> TransactionIdList;
typedef wstring EntryId; // audit trail number for a GL Entry
typedef sequence<EntryId> EntryIdList;
typedef wstring AccountId; // a single GL Account reference
typedef sequence<AccountId> AccountIdList;
typedef wstring EntryType; // a single GL Entry type
typedef sequence<EntryType> EntryTypeInfoList;
typedef wstring PeriodId; // a reference to a user defined period
typedef unsigned short ChartKind;

Here are some interfaces

+ get_all_account_info() : AccountInfoList
+ get_account (acc_id : AccountId) : Account
+ get_multiple_accounts (account_ids : AccountIdList) : AccountList
+ get_all_accounts() : AccountList
+ get_trans_ids (date_range : DateRange) : TransactionIdList
+ get_trans_info_summary (trans_id : TransactionId) : TransactionInfo
+ get_trans_info_by_refs (trans_ids : TransactionIdList) : TransactionInfoList
+ get_trans_info_by_date (date_range : DateRange) : TransactionInfoList
+ get_entry_count_by_type (date_range : DateRange, entry_type : EntryType) : unsigned long
+ get_entry_count_by_account (date_range : DateRange, acc_id : AccountId ) : unsigned long
+ get_transaction() : Transaction
+ get_transactions_by_ids (trans_ids : TransactionList) : TransactionList
+ get_transactions_by_date (date_range : DateRange) : TransactionList
+ get_entries_by_type (date_range : DateRange, entry_type : EntryType) : EntryList
+ get_entries_by_account (date_range : DateRange, acc_id :AccountId ) : EntryList

Now, in contrast here is the OAG Post_Journal, which defines the view of the
"General Ledger" as seen by a broader population of applications integrators:

<!ENTITY % RESOURCES SYSTEM "oagis_resources.dtd">
%RESOURCES
<!ELEMENT POST_JOURNAL_004 (CNTROLAREA , one instance MANDATORY
DATAAREA+ )> one or more are MANDATORY

<!ELEMENT DATAAREA (POST_JOURNAL )> MANDATORY

<!ELEMENT POST_JOURNAL (JEHEADER , MANDATORY
JELINE+ )> one or more are MANDATORY

<!ELEMENT JEHEADER ( (%AMOUNT.DOCUMENT.T; )? , zero or one instances allowed (note 1)
(%DATETIME.DOCUMENT; )? , zero or one instances allowed
(%DATETIME.PAYEND; )? , zero or one instances allowed
GLENTITYS , MANDATORY
ORIGREF , MANDATORY
DESCRIPTN? , zero or one instances allowed
DOCTYPE? , zero or one instances allowed
JEID? , zero or one instances allowed
LEDGER? , zero or one instances allowed
USERID? , zero or one instances allowed
USERAREA? )>

<!ELEMENT JELINE ( (%AMOUNT.ACTUAL.T; ) , MANDATORY one instance in Transactn Currency.
(%AMOUNT.ACTUAL.F; )? , zero or one instances allowed in the FunctionalCurrency.
GLNOMACCT , MANDATORY
BUSNAREA? , zero or one instances allowed (this is like a Quickbooks "Class")
COSTCENTER? , zero or one instances allowed
DEPARTMENT? , zero or one instances allowed
DESCRIPTN? , zero or one instances allowed
DIVISION? , zero or one instances allowed
ELEMENT* , zero or more allowed
FUND? , zero or one instances allowed
GEOGRAPHY? , zero or one instances allowed
GLENTITYD? , zero or one instances allowed
ITEM? , zero or one instances allowed
PRODCTLINE? , zero or one instances allowed
PRODORDER? , zero or one instances allowed
PROFITCTR? , zero or one instances allowed
PROJACTVTY? , zero or one instances allowed
PROJECT? , zero or one instances allowed
PROJRESEL* , zero or more allowed
REASONCODE? , zero or one instances allowed
REF* , zero or more allowed
SALESORDID? , zero or one instances allowed
UNIT? , zero or one instances allowed
WAREHOUSE? , zero or one instances allowed
WORKORDER? , zero or one instances allowed

( (%DATETIME.ACCOUNTING; ) | (ACCTPERIOD , ACCTYEAR ) ) , USERAREA? )>

translation: one of these two things is mandatory:
either a valid "DATETIME.ACCOUNTING" entity as defined in the DTDs,
or a valid ACCTPERIOD , ACCTYEAR pair,.followed by zero or one, optional, USERAREA's.

Pardon my style, of larding the XML DTD with notes and explanations.

The OAG "Post-Journal" is logically, a superset of the OMG's GL Facility. It supports every element of OMG's universal, common denominator specification. Let's examine some interesting points:

1. mandatory fields in OAG include the accounting period (or date) on every row of a journal entry. This is wrong for SMEs. It is improper for a spec like Post_Journal to presume such a deep level in the processing of the General Ledger or its storage. The fact is, none of the small general ledgers support different dates or periods on lines within the same journal entry and this OAG requirement imposes work on coders integrating small business software. (Note, the Post_Journal BOD doesn't even provide a transaction date element in the header, as an option.)

2. arbitrariness. OAG's Post_Journal supports systems which combine a lot of other fields into their GL (project, org. structure, item, product lines, etc.) which is not fatal. However, OAG arbitrarily does not support other systems, which combine other fields (employee, job code, customer, and vendor are common in SME desktop software.) The question is where do you draw the line? If you're going to support systems which denormalize a bunch of facts or dimensions into the general ledger, but not others, then that is not a level playing field. Rather, it is biased in favor of the first-comers' interests, e.g. the ERP vendors.

This is to some degree, a zero-sum game. The number of tags supported within Post_Journal cannot simply be extended endlessly, to include for example, employee, job code, customer, or vendor. There is a natural limit to the number of tags which any given XML parser or computer resource can efficiently manage, and which the development platform can manage, and the human integrator can deal with.

An unconstrained growth of vocabulary would anyways result in non-interoperable systems by including synonyms and near synonyms, and redundancies within different BODs. The objective of non-negotiable interfaces is lost, if humans must negotiate for example, whether they will pass the SALESORDID within the POST_JOURNAL above, or some other BOD. Very much of that sort of thing and you're better off going back to custom interfaces.

I have not determined the extent of overlapping synonyms or other negotiable structures in OAGIS to arrive at any judgment whether it already represents a serious problem to BSPs. But clearly they exist to a considerable extent.

My understanding is the OAG has no "Sunset Law" for field names, or process to provide for orderly retirement of existing elements. There is no "Annual Renewal Vote". To the contrary the OAG operating policy is a substantial commitment to forward compatibility (new BOD versions may include new fields, but will be backward compatible to older BODs). In other words, the OAG's XML DTDs and integration scenarios, on their face, imply a certain approach to the sales journal, purchases, and general ledger which is arbitrary and not necessarily compatible with newcomers' systems. It therefore imposes larger costs on newcomers than incumbents to write software that implements the OAG interfaces.

In contrast, the FCC regularly reallocates frequency spectrum from underused bands for emerging needs. ACORD has a sunset procedure in which any element may be deprecated; it will be supported for one further release of the spec, then may be dropped. (ACORD has over 4500 data elements in 8 schema docs.)

The combination of these systematic, cultural and "process" factors leads me to the conclusion that they are sufficient to discourage any significant presence in the OAG standards process by SMEs. Neither OAG nor SMEs should incur process delays waiting for SMEs to participate, or expend energy inviting them under the present circumstances. This isn't just a matter of "whether OAG is an open process". The question is, "is it, in fact, accessible to small business" and the answer is "NO." Under present circumstances, it is a top-down standard that exists on a "take-it-or-leave-it" basis.

Viewed pragmatically the question for SMEs and Webledger vendors and other BSPs on the internet is not whether they can achieve things thru OAG membership, but rather, what the OAGIS XML vocabulary offers that is of value, and whether the value is sufficient to adopt OAGIS.

Another question for BSPs and SMEs collectively, is whether the wisest choice would be to break off from any existing standard, and create a whole new A2A integration spec. To my mind this would not necessarily be antisocial or wasteful at all. The new XML vocabulary could benefit from the OAG's vocabulary and everything positive it has accomplished, while containing data structures more representative of SMEs. It would be altogether less burdened by legacy, or by hundreds and hundreds of current structures and elements from the ERP and manufacturing integration adopted by OAG. A new, smaller organization could react quicker, operate interactively on the internet, in internet time, in open fashion.

Consider the mechanisms at work in OAG, causing profusion of data elements in a spec like OAG, while the OMG CORBA domain has adhered to such minimalism. Why do developers within OMG, building new systems choose an essential core of C++ code and then add to it? Does the task of interconnecting diverse systems require the opposite? Why did OAG choose this patch panel, a telephone switchboard with a socket for every noun that exists anyplace in the population? Evidently, CORBA developers would not agree to support a large number of data structures even as options, preferring only an irrefutable minimum.

My sense is that the Post-Journal spec has gone astray, by combining domains outside the financial accounting domain. It appears that this characteristic (combining or denormallizing functional domains into a single BOD) is pervasive throughout the 122 BODs. The OAGIS perimeter is too ambitious and needs to be shrunk back to fundamental things which are indisputable and universal. That is a natural consequence when the number of diverse users expands. When you have vast numbers of users, you run into exponential choices at lower branches of the tree which were not visible with only 10 or 20 or 30 participants.

Internet A2A integration doesn't need absolute purity but it needs to be purer than OAG, leaning towards the OMG's philosophy. I can state categorically, that few of the dotcoms are likely to implement more than the core elements of G/L when they emit GL transactions down to the root ledger.

The division of tasks among dotcoms is also going to be in larger aggregates than what you see in ERP systems, and, apparently, it is just different. The whole sales function will be executed by a single provider. There isn't going to be distributed sales systems, or multiple distributed inventory databases on the internet, as a rule. Another example is HR/payroll and miscellaneous functions like timesheets, collaboration and expense reporting will end up aggregated in various ways, by various providers. What this boils down to is: the software modularity, or granularity of OAG may be at an inappropriately fine grain, for integration of the whole wide internet. I am afraid it may assume too much about the functionality occurring at the receiving end of a BOD.

Selling OAG's vocabulary to BSPs and SMEs will be impacted by usability issues. The tag names are bad. Let's face it. They are all uppercase and have needlessly limited their lengths to 10 characters, impacting readability. No BSPs, or SME in the XML integration market, has any 10 character limits whatsoever. Mixed camel case and full English spelling add readability and relieve the developer from memorizing how numerous words have been truncated. Following are some examples:

OAGIS data element:	Why not rename it as:
ABSENCETYP	AbsenceType
ACCTPERIOD	AccountPeriod
ACCTTYPE	AccountType
ACCTYEAR	AccountYear
ACKCODE	AcknowledgmentCode
...	...

The OAG data structures may be lopsided. There are large numbers of entities and attributes. The OAG schemas are very difficult to understand without dedicated tools like XML Authority, because of the use of external entities, multiple DTDs etc. chained in 4 resource files.

In conclusion, OAGIS 6.2 is not the right stuff for internet BSPs.

A new XML vocabulary for integration of Business Services Providers (BSPs) could be quickly produced under the following process:

Guiding principles:

Keeping it simple
Internet time
80/20 rule, rather than seeking perfection
Make the taxonomy and the development process accessible to business domain professionals, who are non-XML non-computer careerists, as a key to making the business professionals work together with developers
Target 500 universal tags admitting most users will extend the schema.
Goal is to nail down the essential core element names, and the level of modularity.

Steps to create a new XML "straw man" draft:

Identify current BSP market segments and a representative sample of BSPs.
Print Screens and document the range of functionality among the sample.
Identify the "minimum common denominator" interface necessary to integrate them into a common theoretical framework or hub.
Identify all existing OAGIS Integration Scenario and BODs for that integration, and identify the fields and segments required therein.
Create new synonyms expressed in mixed case and more appealing to the internet and SME developer, for each OAGIS segment and field required, and drop the MRP elements.
Create a whole new set of schemas which stand on their own like the BASDA eBIS-XML schemas.
Find other schemas such as XBRL and IFX to provide functionality not present in OAG.
Create new OAGIS BODs for the minimum common denominator functionality needed, by drawing on the work of XBRL and IFX.
Rationalize the whole OAGIS Sender in ControlArea, for the internet. Ditch the 10-character limits. Figure out a global scheme for GUIDs such as allocating ranges from an authority, like MAC addresses in ethernet cards. Provide vocabulary for the Subscriber and any other requirements that are universally required by BSPs for authentication, routing and management of business processes on behalf of their subscribers. Figure out a new hierarchy for BSP, Process instead of the OAGIS Component, Task structure.
Send the proposal to the OAG's metadata work group to conform it with any new positions they may have, and OAG vendors and industries. Maximize the equivalence of synonyms etc.
Staff it around the OAG's Convergence committee. Send the Spec to the consortia/sponsors of other leading XML schema such as eBis-XML, xCBL and cXML, and ask them if the structures or elements cause collisions or problems for them.
Produce XSLT stylesheets which provide round-trip XSL transformation to/from OAGIS 6.2
Pay somebody to implement the new XML as dialtone, so that users can test against the reference implementation server.
Get responses from SMEs and BSPs within a fast moving dialog conducted publicly on the internet, such as a web discussion linked with a mailing list, with a rich website with open chat server.

This is what I believe BSPs should consider, at this time. It would be best undertaken jointly by OAG, and the BSP industry association, the IBSI. If the organizations are not receptive, individual BSPs should undertake the work within loose associations.

If neither standards bodies nor collaborations with integration partners can be created to undertake a new XML integration standard, individual BSPs should build their own document level XML interfaces and publish them unilaterally.

Todd Boyle
tboyle@rosehill.net