Offline data app architecture

Ionic Framework
1

Hi.

I’m thinking of upgrading a ionic1 app, from around 2016-2017. Most likely it will be a rewrite.

Now one of the elements in this app was having a offline database, and I used sqllite with a relational model. The app would poll an api and the api send data to the app, where rows would be marked as active or delete, in order to keep the data in the app up to date. Total amount of data is about 1mb.

In hindsight, the data didn’t really have to be relational. This is just a lookup app and don’t need crud operations in the app.

I’m thinking this could be solved easier today.

I’m looking into firestore, to deal with sync of the data, allowing for offline data access.

One thing I’m not sure of though, is how would the data be updated in firestore? Or is firestore overkill?

2

Hi

Firestore is great value and good stuff to work with from a noSQL perspective. I am a happy user.

As to the offline stuff and syncing, it depends on the syncing mechanism. You can have realtime coolness with firestore, but you can also do read/write once. Then it just acts like any endpoint database. It’s just that the API does not need to be REST/http.

As to Firestore persistance feature, this is only intended to cover for flaky connections. NOT offline usage. Not sure where it is documented, but one of the many cool videos by Firebase it is really stressed not to use it for that purpose.

I did work on an offline first app myself but reverted to realtime online anyway (with persistence on) as I need the users to be online for auth anyway. Setting up syncing manually can become quite daunting btw. I needed both ways, which in hindsight is horror.

But if it is only about taking regular snapshots from a server and then plant it in the app, it will be easy. A simple pull of data and then overwrite the data in a local storage.

As to local db, you can also go for indexedDB. Especially if your users are on android. And then even use PWA, if you are not relying on lots of plugins. 1Mb on IndexedDB on iOS is even small. Deploying as PWA could be beneficial for your case?

I am a big fan of Firebase, but there are many other, similarly priced, solutions that do not require prying eyes of bigtech.

And looking at your other q - how to get the data in firestore. That depends where it comes from. Another app or from another server? If it is another server, you likely need Firestore functions. If it is an app you build yourself, then it is just an admin app next to a user app.

Whether you should use the same structure (marking for delete or not) I’d say, consider rethinking the data-model for the end user app - focussing on fast UI. Firestore can handle mutations pretty well under the hood and refresh what is needed. And 1Mb isn’t a lot.

3

Thank you so much for a complete answer. Really appreciate it. I will have to read this a couple of times to process it :slight_smile:

The users of the app, might be in a place with poor connection, that´s why I am thinking of reducing the amount of data sent to the app for updates. I´ve earlier implemented a delta load of the data, so only getting the data since the last time since the app was openend. Would be nice with automatic update.

The data comes from my own API. Solution was once the app opened up, it would query the api to get the latest data. Also would delete data that was no longer relevant.

1 Like
4

You either make your own mechanism for polling delta, for instance a list of all changes after a timestamp and then process at app level

Or let firestore handle that for u. All dependent the data and change frequency you may aftually want to run realtime update using persistence for flaky connections -

Likely being even easier on bandwidth and battery compared if u do this yourself

Maybe a bigger need is a caching feature that loads the last known good data from local storage at app launch, until connection is established/requested. Which then updates that cache next to the state in the app

The only q i have here is how to get the stuff from your api in firestore. What is your api endpoint? Rest interface?

5

Again, thank you.

The API is currently a REST, but could potentially push data to firestore.

I´ll have to rewrite this API as well (its most likely going to be a spring boot appliation).

But I don´t need real time update - once a day is more than enough.

6

Hi
If u control the api endpoint then I wonder why u would want firestore in between

I assume u can control authentication too, and all other security elements of the api, costs, reliability etc.

So then the api needs to provide the delta as per timestamp using a http get. That simple

7

Yes, I do control the endpoint. And will have to rebuild this, as it’s on an old platform. Most likely using a SpringBoot application for this.

And I am not sure If I really need the Firestore as you say. That’s basically why I opened up this discussion :slight_smile: Trying to wrap my head around how I should refactor it.

What I would like to have, was a easier way of updating the app and keeping the data in the app synced with the REST API. I was wondering if Firestore would make it easier for me.

Another issue is that the endpoint is built up with on a table-by-table basis, but perhaps what I am really looking for is a better way of sending this data, and I was hoping Firestore (or perhaps something similar) could ease that transition for me, as in help me with keeping the data synced.

There is no authentication, it’s totally open and that’s fine.

Thanks again Tommertom. Really appreciate your answers and the discussion.

1 Like
8

If it can help, I’m using Dexie (https://dexie.org/) in a Angular 10 project.

It’s a nice library that uses IndexedDB for storing offline data as objects, so you don’t need stringify objects and store in localstorage.

I’ve not tested this solution in Ionic App, but you can try…

9

Sounds pretty similar to Ionic Storage.

10

Yes, it is.
But Dexie is just for IndexedDB and you have more control about Interfaces and table contracts (fields).
You can do a versioning of your database as well.