Persistent no-SQL storage in Ionic2 - tutorial


#1

Hi All,

Here’s how I built a LokiJS database with LocalForage for persistent storage. All feedback is welcomed.

My Requirements / App Wish List

  1. a no-SQL database
  2. long-term data persistence
  3. simple, legible code and as few adapters as possible
  4. platform agnostic

That 4th point is the kicker. That knocks out IndexedDB and the like. For more details, see CanIUse.Com. I began writing for PouchDB but found that it was far too slow, and saving the database to a server was much more burdensome than their streamlined tutorials show. I also wanted more granular control of its cloud sync process.

The main reasons I opted for LokiJS

  • simple, familiar JavaScript objects
  • good documentation on lokijs.org
  • in-memory architecture
  • ability to store full DB as a JSON token (awesome for small DBs!)
  • microscopic footprint

The main reasons for writing this tutorial

  • LokiJS and LocalForage are great tools
  • I’m yet to find an Ionic-centric tutorial on them online
  • I need to improve my TypeScript, so this is a good exercise for myself

So let’s begin…!

My Environment

June 22, 2016 Cordova CLI: 6.1.1 Ionic Framework Version: 2.0.0-beta.9 Ionic CLI Version: 2.0.0-beta.25 Ionic App Lib Version: 2.0.0-beta.15 ios-deploy version: 1.8.6 ios-sim version: 5.0.8 OS: Mac OS X El Capitan Node Version: v5.7.1 Xcode version: Xcode 7.3 Build version 7D175

Generating the skeleton code

In your terminal, type the commands…

ionic start LokiDB blank --v2
cd LokiDB
ionic platform add ios
ionic platform add android
npm install lokijs
npm install localforage

All of our code will be modified in app > pages > home > home.html and home.ts.

Coding Strategy

  1. add LokiJS without persistence
  • add interactive elements to our LokiJS database
  • add LocalForage to gain persistence

Adding LokiJS without persistence

  1. In home.ts, just under the import statements, add

    declare var require: any;
    var loki = require(‘lokijs’);

  • Inside the HomePage class, we need to declare 2 objects: one for our database and one for its collection of documents

      db: any;                        // LokiJS database
      robots: any;                    // our DB's document collection object
    
  • Let’s set up these objects inside the constructor

      this.db = new loki('robotsOnTV');
      this.robots = this.db.addCollection('robots');
    
  • Next, we’ll insert a few documents (for those who aren’t used to no-SQL databases … me included… a document is just an object held by the database). We’re using JSON style insertion because that’s how LokiJS receives the data. Don’t worry about making TypeScript interfaces; that will only add to code we’d need to write.

      this.robots.insert({ name: 'Bender', tvShow: 'Futurama' });
      this.robots.insert({ name: 'Rosie', tvShow: 'The Jetsons' });
      this.robots.insert({ name: 'K1', tvShow: 'Dr. Who' });
    
  • The final thing to do in the TS file is to add a helper function. We want the HTML file to display these results, but *ngFor will not iterate over custom data types, so we’re going to write a simple, generic object-to-Array function:

      convert2Array(val) {
          return Array.from(val);
      }
    

This is how your home.ts should look:

import {Component} from "@angular/core";
import {NavController} from 'ionic-angular';

declare var require: any;
var loki = require('lokijs');

@Component({
    templateUrl: 'build/pages/home/home.html'
})

export class HomePage {
    db: any;                        // LokiJS database
    robots: any;                    // our DB's document collection object

    constructor(private navController: NavController) {
        this.db = new loki('robotsOnTV');
        this.robots = this.db.addCollection('robots');

        this.robots.insert({ name: 'Bender', tvShow: 'Futurama' });
        this.robots.insert({ name: 'Rosie', tvShow: 'The Jetsons' });
        this.robots.insert({ name: 'K1', tvShow: 'Dr. Who' });
    }

    convert2Array(val) {
        return Array.from(val);
    }
}
  • Lastly, let’s get the HTML ready. Delete everything inside the <ion-content> tag of home.html, and replace it with this:

      <!-- list all database elements -->
      <ion-card *ngFor="let robot of convert2Array(robots.data)">
          <ion-card-header>
              {{robot.name}}
          </ion-card-header>
          <ion-card-content>
              {{robot.tvShow}}
          </ion-card-content>
      </ion-card>
    

Adding interactive elements to our LokiJS database

  1. Inside home.ts, add 2 variables for user input

    robotName: string;
    robotTVShow: string;

  • We’ll add in support to insert and delete from the database as well:

      addDocument() {
          if (!this.robotName || !this.robotTVShow) {
              console.log("field is blank...");
              return;
          }
    
          this.robots.insert({ name: this.robotName, tvShow: this.robotTVShow });
    
          // LokiJS is one's-based, so the final element is at <length>, not <length - 1>
          console.log("inserted document: " + this.robots.get(length));
          console.log("robots.data.length: " + this.robots.data.length);
      }
    
      deleteDocument($event, robot) {
          console.log("robot to delete: name = " + robot.name + ", TV show = ", robot.tvShow);
    
          // $loki is the document's index in the collection
          console.log("targeting document at collection index: " + robot.$loki);
          this.robots.remove(robot.$loki);
      }
    
  • Let’s add one more card to home.html

      <!-- add items to LokiJS database -->
      <ion-card>
          <ion-card-content>
              <ion-list>
                  <ion-item>
                      <ion-label floating>Robot Name</ion-label>
                      <ion-input clearInput [(ngModel)]="robotName"></ion-input>
                  </ion-item>
                  <ion-item>
                      <ion-label floating>Which TV Show?</ion-label>
                      <ion-input type="text" [(ngModel)]="robotTVShow"></ion-input>
                  </ion-item>
              </ion-list>
          </ion-card-content>
          <ion-card-content>
              <button (click)="addDocument()">Add</button>
          </ion-card-content>
      </ion-card>
    
  • Finally, we need to allow for document deletion, so let’s change the original card so we have a Delete button:

      <!-- list all database elements -->
      <ion-card *ngFor="let robot of convert2Array(robots.data)">
          <ion-card-header>
              {{robot.name}}
          </ion-card-header>
          <ion-card-content>
              {{robot.tvShow}}
              <button (click)="deleteDocument($event, robot)">Delete</button>
              <button (click)="saveAll()">Save All</button>
              <button (click)="importAll()">Import All</button>
          </ion-card-content>
      </ion-card>
    

Long-term Storage

Ready for the last part? Let’s go!

We’re going to allow for saving to file and importing from that file. For more info on how LocalForage prioritizes storage, see http://mozilla.github.io/localForage/

  1. home.ts needs a localForage object. Add this just below your ‘var loki…’ code:

    var localforage = require(‘localforage’);

  • Add in functions for saving the database and retrieving it. LocalForage uses key-value maps, and since we’re only interested in saving 1 value (the entire database), we’ll hard-code our key as storeKey.

      saveAll() {
          localforage.setItem('storeKey', JSON.stringify(this.db)).then(function (value) {
              console.log('database successfully saved');
          }).catch(function(err) {
              console.log('error while saving: ' + err);
          });
      }
      
      importAll() {
          var self = this;
          localforage.getItem('storeKey').then(function(value) {
              console.log('the full database has been retrieved');
              self.db.loadJSON(value);
              self.robots = self.db.getCollection('robots');        // slight hack! we're manually reconnecting the collection variable
          }).catch(function(err) {
              console.log('error importing database: ' + err);
          });
      }
    
  • In home.html, we’re going to hook up the new storage functions to 2 new buttons. Next to our “Add” button, include these:

              <button (click)="saveAll()">Save All</button>
              <button (click)="importAll()">Import All</button>
    

That’s all it takes to build a persistent, no-SQL database in Ionic 2!

Thanks for reading,
Ryan Logsdon


PouchDB with Quick Search plugin throws 'Error: Cannot find module "./lib/primes.json"'
LokiJS using IndexedDB adapter on a Windows 10 platform
Why is ionic2 using websql for the browser when it has been deprecated?
#2

Excellent tutorial. Thanks !

I come from Meteor and I used minimongo, which was a partial implementation of MongoDB as an in-memory db for the browser. Minimongo was also reactive, so we could directly display a query in the template and the template updates automatically when the query result changes.

My quest since 2 weeks is to find something like that in Angular2. I’m looking for a No-SQL in-memory DB that is observable, so I could use the Angular2 async pipe in the templates.

I was hoping LokiJS had that but it doesn’t :frowning:


#3

Great tutorial @ryanlogsdon !

I can’t “require” anything with TS, how do you manage to use it ? Usually i’m using “import”…

Thank you !

EDIT : find my answer :

declare var require: any;


#4

Copy my home.ts file verbatim. You don’t have this line in yours. :slight_smile:


#5

I like the way you’ve written this: clear, concise and practical. Easy to read because it cuts out all the fluff and tells me only what I need to know. I’d like to attempt writing a tutorial like this and would use your format for inspiration!


#6

I really appreciate that!!


#7

Hello,
I need help in following this tutorial.
ionic info output:
Your system information:

Cordova CLI: 6.3.1
Ionic Framework Version: 2.0.0-rc.0
Ionic CLI Version: 2.1.0
Ionic App Lib Version: 2.1.0-beta.1
ios-deploy version: 1.9.0 
ios-sim version: 5.0.8 
OS: Mac OS X El Capitan
Node Version: v6.6.0
Xcode version: Xcode 8.0 Build version 8A218a 

My whole home.ts code:
import { Component } from ‘@angular/core’;

import { NavController } from 'ionic-angular';

declare var require: any;
var loki = require('lokijs');

@Component({
  selector: 'page-home',
  templateUrl: 'home.html'
})
export class HomePage {
  db: any;
  robots: any;

  constructor(public navCtrl: NavController) {
    this.db = new loki('robotsOnTV');
    this.robots = this.db.addCollection('robots');

    this.robots.insert({ name: 'Bender', tvShow: 'Futurama' });
    this.robots.insert({ name: 'Rosie', tvShow: 'The Jetsons' });
    this.robots.insert({ name: 'K1', tvShow: 'Dr. Who' });

  }

  convert2Array(val) {
    return Array.from(val);
  }

}

As you can see, I already declare require as any but when doing ionic serve I got this error on the developer console:
require is not defined

Can somebody help me? :sweat:


#8

I have similar issue. I’m guessing this has something to do with the move to using Rollup.

Until this issue gets sorted I just included the lokijs.min.js file directly into my index.html page. Loki becomes a global object, but at least it got me up and running.


#9

Hi @s7ven,

Could you explain where you saved the lokijs.min.js file, and exactly how you imported it in the *.ts file?

Thanks,
Ryan


#10

Hi.

Anything you put in the /src/assets/ folder will be copied across to the www/assets/ folder.

So add this to your index.html

<script src="assets/js/lokijs.min.js"></script>

Declare loki object in your .ts file in a simple way to stop typescript whining

declare var loki: any;

Then you can instantiate a loki object like

myLoki = new loki( ...

hth.


#11

Hi @s7ven,

I still get the error “loki is not defined”.

This is the entirety of my *.ts page…

import { Component } from '@angular/core';

declare var loki: any;

@Component({
    templateUrl: 'hello-ionic.html'
})

export class HelloIonicPage {

    public db: any;                        // LokiJS database
    public robots: any;                    // our DB's document collection object

    constructor() {

        this.db = new loki('robotsOnTV');
        this.robots = this.db.addCollection('robots');

        this.robots.insert({ name: 'Bender', tvShow: 'Futurama' });
        this.robots.insert({ name: 'Rosie', tvShow: 'The Jetsons' });
        this.robots.insert({ name: 'K1', tvShow: 'Dr. Who' });

    }
}

I downloaded the minified LokiJS code and saved it to “src/assets/js/lokijs.min.js”.

And I’ve added the tag you mentioned to index.html.

Any thoughts why loki is still not recognized.

Thanks,
Ryan


#12

check your paths and look in folder to make sure it’s in there. sounds like it hasn’t been loaded.

/src/assets/js/loki.min.js

if file is loading ok, then use console to check what object is in global namespace. it wouldn’t surprise me if the actual name has changed from version to version. Just start typing

window.loki