For a number of reasons, neither compliance nor interoperability testing can provide complete assurance that two software applications will work together seamlessly at your specific installation.

For example, if a specific utility has chosen to use non-standard formatting for customer identifiers or has chosen to store those identifiers in a non-standard data field, the link between customer records in a customer information system and an automated meter reading system could be jeopardized.  The MultiSpeak interface might work properly and pass interoperability testing provided the customer identifier is stored as anticipated by the two vendors.  However, the fact that the customary data field is missing at a specific utility could cause the link to fail only at that utility.  Such a failure would not be a reflection on the generic suitability of the MultiSpeak interface.  Rather, the utility’s choice to use non-standard data is the root cause of the problem.  In this case, the standard MultiSpeak interface could still be customized slightly to work for this utility or the utility could decide to change their data structures to match the anticipated condition.

There is another situation where two products that have passed interoperability testing might work properly at one utility and fail to work together as expected at a different utility: poor data quality.  For example, if customer telephone numbers are not accurate in the customer information system, an outage management system or an interactive voice response system that relies on those telephone numbers might give unacceptable results, even though the telephone numbers were sent accurately from the CIS to the other system.

If it is necessary (i) to document the level of integration functionality, (ii) to determine the root cause of failure of two software applications to work together as expected, or (iii) to determine what customization is required to achieve the expected result in non-standard situations at a specific utility, site conformance testing is required.  Utilities may request that an independent party perform site conformance testing on their behalf.  In addition, utilities may find it desirable to require site conformance testing as an acceptance criterion when purchasing new software products.