I once worked on a platform where you could buy ringtones, wallpapers, jokes etc by SMS pre App Store and play store.
Every night the platform would kick off a process to claim money from the telco using some key which was tied to an SMS the user had sent.
That process was responsible for more than $1 million a month, but it was buggy as hell.
Half of the billing attempts would fail, the retry logic was buggy, processes would crash so you would just start them again without proper semantics to ensure you don’t bill people twice.
After a while there were so many retries outstanding that you couldn’t get through them overnight, so we would try in random orders and we would have the process running from 3 days ago whilst try close off today’s bills.
I’m sure people were billed incorrectly and a lot of money was left on the table from users who should have been billed. We never really heard a lot of noise about this either.
So I’m sympathetic to this process and glad that Apple and Google have improved the confidence around mobile commerce.
(Googler, opinions my own. I also do paymentsy things.)
Carrier Billing has definitely gotten better and easier. From the Google Play side of things, there is an API that is used to help with things (no public docs I can find sadly).
Not just back in the day, I have at least a couple of customers of which I know they are losing money because of billing issues.
Why? Because most accounting software is horrible and have minimal to no third party integration options. Many of them don't have an API, so the only option you're left with is manually (!) importing some XML structure every day. The logic is usually very limited so stuff like record mutations are often silently ignored during the import.
The accounting software world is evil. Deliberately engineered vendor lock-in, very limited third party integrations, ridiculous charges for 'custom' implementations, etc. I once build a migration tool for a well known accounting/ERP suite, and my partnership with them was immediately terminated due to 'breach of terms'.
Google themselves stole $75 million from Adwords customers by not coding up a particular kind of refund and ignoring it for decades. It is likely a similar thing is happening with banned account balances.
That’s interesting! I’ve often remarked that it’s a giant competitive advantage for huge companies like Comcast to be able to net however many millions of dollars from billing “mistakes” that never get contested. On the one hand, I’m willing to concede they are mostly probably honestly mistakes and they only make up some tiny percentage of all their transactions, on the other hand, it probably adds up to a decent sized company’s total revenue every year that they basically get just because they’re so giant they can screw up a ton of transactions and plausibly claim it’s not fraud.
There's a lot of banks in various countries whose standard “payment API” is uploading a CSV file with a list of account numbers to an FTPS or SFTP server nightly. And guess what happens if you upload the file twice by accident…
I would be more than happy if they just use FTPS/SFTP. I've had enough nightmare working with few that require proprietary Managed File Transfer software (aka MFT) e.g. HULFT, AS1/AS2/AS3 clients.
Every night the platform would kick off a process to claim money from the telco using some key which was tied to an SMS the user had sent.
That process was responsible for more than $1 million a month, but it was buggy as hell.
Half of the billing attempts would fail, the retry logic was buggy, processes would crash so you would just start them again without proper semantics to ensure you don’t bill people twice.
After a while there were so many retries outstanding that you couldn’t get through them overnight, so we would try in random orders and we would have the process running from 3 days ago whilst try close off today’s bills.
I’m sure people were billed incorrectly and a lot of money was left on the table from users who should have been billed. We never really heard a lot of noise about this either.
So I’m sympathetic to this process and glad that Apple and Google have improved the confidence around mobile commerce.