Getting from 99.9% reliability to 99.99%

I have an Ionic 5 (Angular) app. It’s a journal app where you have daily entries containing text and images.

Currently whenever a user tries to save the data it works around 99.9% of the time as expected. I’m trying to get the reliability to 99.99% or better. When 10,000 people are tapping “Save” every day, even just 0.1% is a handful of people who experience a problem and then email me about it. :upside_down_face:

Currently when I save in the app it:

1. Saves to our primary Rest API (Laravel-based)
2. Saves to our backup Rest API (separate non-Laravel code, hosted separately from primary API)
3. Saves to the phone database (SQLite using Ionic Storage)

So in cases where #1 can’t connect or doesn’t work for some reason, but #2 is able to, our backup system updates everything as needed. This backup API is fairly new and it has helped some.

Sometimes a user can’t connect to both our primary and backup APIs, despite claiming to have a solid Internet connection. I can see in our API system logs and logs that their app never even tried, which leads me to believe they had no Internet, but users often disagree about that. In that situation, the data is usually saved locally to their phone and it tries again next time they tap Save or open the app again. This method has also helped some.

There are times when both #1 and #2 fail, and then #3 doesn’t seem to work either as at some point the app is no longer able to read the data it saved to SQLite. It’s as if everything just decides to break and the data just disappears. I used Sentry to help catch and track errors, and nothing has shown up that I can pinpoint this to.

I think I can help this by adding a 4th way to save data:

4. Save JSON data to the file system using Ionic’s Filesystem API, in case the SQLite database is somehow empty or inaccessible

And my hope is that this will also help. My guess is that it will help, yet won’t solve everything.

… But let’s take a step back. This is a constant thorn in my side. I feel like I must be missing something fundamental. Is it normal to need so many backup systems in place to create a reliable app for something as relatively simple as a text and image based journal app? Is it usual for production systems with thousands of daily users to have a handful every day that fail to process correctly?

How would you tackle this situation? My thought is that I might need to hire some expert Ionic mobile devs to look things over. With only a couple years of Ionic experience, I consider myself at an intermediate level.

I like this approach as alternative concept:

Never used it because of non-rest backend i needed

Maybe nice to log connnectivity and http errors?

1 Like

Hi Tom, thanks for the link. Simon is an amazing resource for Ionic. The courses from Josh Morony and Simon are where I’ve learned 99% of what I know about Ionic from, haha.

I use a similar technique from that article for my method of saving data locally to the phone and then retrying it later on if it couldn’t connect to the APIs the first time.

However, I don’t log connectivity or HTTP errors to the phone. I only try sending those to Sentry, but those would probably fail too if they don’t have a connection. I like your idea of saving them locally. That might give me more debugging info to work with. Thanks for the tip.

1 Like

I built myself a simple logger service that stores logs locally and when really online, dumps it to the server

Not very difficult

1 Like