Standard Service Offering

Hello all! In March 2017, DIAL held an expert advisory workshop with a number of providers and users of mobile communications services—in particular Voice, SMS, and USSD.

One of the outputs of the workshop was a set of recommendations to improve access to these services and to ease the business and operational relationship between consumers of these services (often implementing NGOs) and providers.

One of the recommendations was to develop a standard definition of the core services that can be used by providers to advertise their capabilities, and taught to consumers (here is the full output of that workshop).

As part of these standard definitions, we also brought up the idea of defining a basic minimum set of services that providers could offer to support the work of International Development and Humanitarian organizations.

The hope is that with a defined minimum Standard Service Offering based on clear service definitions, service providers can better advertise their services, and consumers can better understand a simple set of technical services that are available to support their work.

DIAL would like to facilitate the definition of the Standard Service Offering, and once defined, help service providers implement the offering; help educate implementing organizations; and help market these services—connecting NGO clients to providers.

But first we need to define the offering and each component service!

1 Like

Rather than start by listing and defining every possible service, I think it will be easier/more efficient, to start by debating the minimum/least-common-denominator set of services needed to support common implementing NGO use cases—and then define those.

To kick off the conversation, we had a couple quick white-board conversations at DIAL and have put together a straw man proposal.

For people not familiar with American business jargon, a straw man proposal is a starting point for a discussion. It is a rough draft intended to be argued with, changed, torn apart and/or improved.

In thinking about this straw man, we limited our thinking specifically to services provided via a machine-to-machine (API) interface as we did not consider application-level functionality minimum.

We thought about this primarily from a client perspective: what minimum services would an NGO need to add communications services to their software systems and what services would companies that provide application-level-services to NGOs need to connect their applications to communications services.

For example what would UNICEF need to launch U-Report or WVI to launch ttC? What would FrontlineSMS, TextIt, engageSpark, or InSTEDD need to launch their applications in a new market?

With this in mind, our straw man suggests services in the following categories:

  • SMS
  • USSD
  • Voice
  • Insight
  • Administrative

And because we are not great at strict prioritization, we also labelled services as:

  • Core—the bare minimum set
  • Recommended—beyond the bare minimum, but really good to have
  • Advanced—what some providers might want to aspire to!

Here is what we came up with (see link to view and edit/comment in Google Sheets after the image):


https://docs.google.com/spreadsheets/d/1rrPgAaef0OJZFnPsEBan2Rt3LjaZhQfPaKBdbOrUV_I

1 Like

Questions:

  • What do you think of this framing: the categories of services? Grouping them by Core, Recommended, and Advanced?

  • Can we eliminate any of the service categories?

  • Are there missing service categories that should be included? That have Core services within them?

  • What about the service details we are suggesting? These are some of points we thought might be controversial:

    • In Core SMS we are suggesting string-based SMS APIs where the server handles charset conversion and concatenated messages.

    • Core Voice services are essentially “headless IVR”—analogous to Africa’s Talking, Twilio, and Nexmo’s basic voice offering.

    • Under Administrative, we are suggesting programmatic Long Code Provisioning (even if the provisioning has manual steps, providers can offer APIs)

1 Like

Happy to be part of the discussion. The original PDF (Analysis) addressed payments in addition to SMS, Voice and USSD. It seems the Standard Service Offering has dropped payments - will this be discussed elsewhere?

Thanks,
Luke

1 Like

Hi @Luke_Kyohere, that’s a really good question!

When we were thinking about the straw man in terms of a minimal set of services, we debated what big categories of services to consider ‘minimal’.

We decided to go with Communications Services as a starting point—though to be honest, we didn’t debate it too much, because it’s just a straw man.

Given that you are an expert in payments services, would you be willing to propose an analogous straw man for payments?

I added a ‘Payments’ column to the straw man Google Sheet

What to take a shot at filling in Core, Recommended, and Advanced payments services?

@Diana @kailikfoh – any thoughts here?

Considering that many development organizations regularly disburse high volume, low value payments to millions of recipients for program/operational expenses, I believe payments does qualify to be in the list of least common denominator services…or comes a very close second to communication services!
And it is great to have experts such as @Luke_Kyohere here to help us identify what the industry standards for mobile payment services are, and the categorization of these from core-advanced based on NGO needs.

Hi everyone,

overall, I wonder how compliance to this SSO would look like—is there room to comply with SMS and payments, but not with Voice and USSD? Is it about a provider reaching a level overall and communicating that to potential partners? For NGO’s I’d probably go by use-cases: A provider certifies/complies that their API support a bunch of use-cases as defined by the SSO. That may make it easier for NGOs to decide on a partner. “High-level aggregators” like us (engageSPARK) would probably be more interested in just seeing which Core SMS services are being complied with, etc.

Re payments: Agreed, was also not sure why it’s missing, especially since airtime topup and reverse billing were mentioned. I’d probably move “airtime distribution” to the payments column for a start. And if it’s part of “payments” then it would be basic, right? (Is it core in terms of that category, or core in terms of NGO’s capabilities to run programs?)

USSD: I’m curious to hear from others how relevant that is going forward for the programs that NGOs launch.

Admin/Utility: I understand that “Message cost calculator” is something important to have, but does it need to be an API? If someone wants to launch a program, isn’t that a one-time UI tool? Similarly with “Message page count calculator”—why as an API?

Speech-to-text: What would compliance mean here? I’m wondering about the language diversity, and the capabilities to recognize something other than the most common languages in NLP toolkits (English, Spanish, Chinese, …) Looks like a can of worms. Omit?

Hosted flow/tree logic: From the subtext “(single asyncronous delivery on session close/timeout)” I read this as receiving after-the-fact notifications about what happened, in the SMS/Voice/USSD campaign, maybe via Webhook. Is this correct? If yes, where would one find behavior that allows to disburse airtime in response to completion of a survey? Is that logic part of “Reverse-billing” ?

“Specify # rings before pickup”: More minimal would be config to simply not pick up.

“automatic charset handling” is part of Core’s “String-based Two-way SMS”, but it’s also sort of a Recommended feature in “Automatic character substitution”? Maybe I’m misunderstanding?

Unfortunately, in countries where USSD is available as a service, I think it will remain relevant. It’s a terrible protocol, susceptible to lots of timeouts but universally accessible. For quick data capturing we’ve found it to work better, because the feedback loop is instant, and cheaper, because capturing the same reliably via SMS often requires multiple (costly) interactions. SMS and USSD will likely remain important to provide services to those who need it the most. Unfortunately I predict both will become increasingly expensive over time as their market share diminishes.

Couldn’t agree more. From stakeholder feedback so far, inclusion of mobile money services would be appealing to a number of larger NGOs, and has the added benefit of being a core focus area for telcos.

Typical scenarios for mobile money are Business to Customer, Customer to Business, Customer to Customer, Business to Business

Feels like Business to Customer is the most popular application e.g. humanitarian cash disbursements, conditional cash transfers for mothers, incentive payments for farmers / community health workers. However, I’m sure we can think of more use cases in the other three.

Open to suggestions @Luke_Kyohere as to how we break this down to the next level of detail!

overall, I wonder how compliance to this SSO would look like—is there room to comply with SMS and payments, but not with Voice and USSD?

you make a valid point. it may be hard to define core by channel, given there are plenty of SMS-only services (your standard bulk messaging campaigns) out there, and other voice-only services (a medical hotline) etc. Perhaps the SSO is about compliance to certain minimal capability within a channel (e.g. in SMS, the ability to do delivery reporting, among other things). In short, I would not see a problem with having a SMS-only compliant aggregator and that in some cases you would prefer to work with a specialist player that focuses on one channel (and perhaps has direct MNO connections) rather than a generalist aggregator that can deliver multiple channels but who perhaps executes through another 3rd party aggregator.

However, apart from channels, perhaps there are other services which are truly common across all M4D services (e.g. short / long code provisioning, reverse billing).

@jwishnie for SMS SSO, I’m curious what motivated for synchronous delivery reporting as a core offering? The only reliable delivery reporting mechanism I’ve seen is asynchronous since the underlying protocols themselves are asynchronous.

I would vote to move async delivery reporting to core instead of recommended, dropping sync delivery reporting entirely.

In what scenario is low level UDH needed? Since automatic character substition & handling, message splitting, and concatenation is already handled I don’t see (nor have had) a need for it.

Often our biggest problem has been verifying that a gateway is actually up, running and responsive. From a technical perspective, I would suggest this needs to be core. Working with aggregators + MNOs often means having to guess where a problem is because the health of systems is not transparent. The SSO should be able to give insight or at least some kind of signalling around backlogs and upstream throughput. A message delivery delay at the MNO is often indistinguishable from queuing problems at the aggregator level leading to lots of confusion, blame shifting, and time wasted as a result.

This was @dmccann’s proposal. I think the reasoning is that it’s generally easier to implement a synchronous API, though async delivery reporting does better match the underlying protocols.

I’m happy to drop Sync and move Async into Core.

@dmccann what do you think?

I think this would be a pretty edge case where, for example, you want to hand-build a message in a unique encoding (‘artisanal’ sms). I’d be happy to drop this unless there is an outcry,.

1 Like

@smn Can you propose a set of monitoring/health/heartbeat functions you’d like to see in core?

This makes sense to me–though I see some utility of having a multi-channel “base” or standard of some sort. Specifically in making it easy to explain and market to less technical customers.

Maybe we define ‘core, recommended, advanced’ per category (SMS, Voice, USSD, Payments etc…) and also consider defining standard ‘bundles’ e.g. “communications service bundle” ?

@muratk the motivation for an API is to enable user-level applications to provide cost/volume estimates to the end-user.

These apps can of course include their own encoding & splitting logic, but that creates the possibility that the API provider and app count messages differently creating error.

It’s a bit like the “price-the-cart” problem where it’s best to have a single code path.

For example, I’ve seen some SMS systems use a simple logic for charset encoding and CSM calculation as follows:

  • try to encode message buffer in GSM 03.38
  • If succeeds, divide number of characters by 160 to determine number of SMSs
  • If fails, divide number of chars by 70 to determine number of SMSs

But a more sophisticated system would encode each individual message in a CSM as efficiently as possible, so that if a non-GSM char appeared once, say in the first 70 chars, the first message would be Unicode encoded, but the rest of the buffer would go as GSM.

If the underlying service provider uses one approach and the app another, counts will be off.

@muratk I think you caught a typo here. I believe we meant text-to-speech, not speech-to-text. @dmccann can you confirm?

Of course your question on languages still applies.

My gut would be to leave the specific languages out of any proposed standard, and maybe simply say something like “Text-to-Speech for 1+ languages”

Of course any provider, when describing their own offering, would want to list the languages they support.

@muratk – if the most basic APIs for handling SMS/USSD/Voice conversations is some kind of callback system (like web-hooks), this would be the next level of hosted functionality where a provider can run an entire interaction without needing to call webhooks–perhaps by implementing the RapidPro Flow specification .

The idea being that clients of the API could provide in advance any necessary assets (e.g. audio files for IVR) and rules such that the provider can run the interaction and provide the results to the client.

Honestly, this might never make sense for an API provider to offer. It may be a feature that makes most sense for an app provider like engageSpark to handle at the user level.

Hence ‘advanced’

I would worry that putting this in core would have strange unintended consequences. I’ve seen aggregators implement fake (and immediate) MT delivery reports for MNOs that didn’t supply delivery reports because that’s what a contract stipulated.

I’m struggling to define a set of criteria that’s comprehensive enough to be useful as what is possible depends on the underlying protocols used. Here’s a very incomplete attempt:

  1. For protocols that implement an underlying connectivity signal, expose an API that provides a view of the last n signals received and map those to standardized indicators.

Example: SMPPs enquire_link PDU is sent at regular intervals. An API should be able to map this to a standardised indicator that represents an established connection.

  1. For protocols that implement a stateful connection, expose an API that maps the state of the connection to a standardised indicator.

Example: An SMPP connection being bound in rx, tx, or trx mode can be mapped to indicators that indicate that the authentication has succeeded. Absence of it means a disconnection.

  1. For systems that relying on internal queing, expose an API that gives insight into queue sizes and growth / drain rates. Junebug provides some support for this.

  2. For stateless connections (which most USSD protocols are) provide an API that the succes rate of n most recent USSD replies (if the protocol is asynchronous) and the n most recent USSD requests received.