forEach loop with synchronous promises?


#1

I’m struggling to get my head around promises and I could really use some help.

This is my desired behaviour for my app:

  • User can create multiple data records and save them in local storage (when no connectivity present)
  • When connectivity is present, user can click the “Upoad” button, which will grab all local storage records and for each record it will do the following:
    ** Push data item to existing online database
    ** Delete data item from existing local storage records

For example, if I had 3 local records then I would want the code to execute things in the following order:

  • Display “Uploading” loading dialogue
  • Upload data item #1 to online DB
    * [Wait for upload to complete]
  • Delete data item #1 from local storage
    * [Wait for delete complete]
  • Upload data item #2 to online DB
    * [Wait for upload to complete]
  • Delete data item #2 from local storage
    * [Wait for delete complete]
  • Upload data item #3 to online DB
    * [Wait for upload to complete]
  • Delete data item #3 from local storage
    * [Wait for delete complete]
  • Dismiss “Uploading” loading dialogue

Below is an example of the code i’m trying to use. For the purposes of this forum post, local storage is simply “local-db” and online storage is simply another instance of local storage called “online-db”.

uploadLocalData_Promise(){

	return new Promise((resolve, reject) => {

	  this.getData_Promise('local-db').then((localDataArray: any) => {
		localDataArray.forEach((localDataObject) => {
		  
		  this.setOnlineData_Promise(localDataObject).then(() => {
			console.log("Data record #" + localDataObject['id'] + ": Online record created");
			
			this.removeLocalData_Promise(localDataObject['id']).then(() => {
			  console.log("Data record #" + localDataObject['id'] + ": Local record removed");
			  
			}, (err) => {reject(err);});
		  }, (err) => {reject(err);}); 

		});
		resolve(true);
	  }, (err) => {reject(err);});
	});
}

Given the asynchronous nature of promises, the above code generates the following console output:

  • Data record #1: Online record created
  • Data record #2: Online record created
  • Data record #3: Online record created
  • Data record #1: Local record removed
  • Data record #2: Local record removed
  • Data record #3: Local record removed

I understand why it does this (i.e. due to asynchronous nature), but is there any way I can get each promise to execute synchronously? I’ve tried using Promise.all on the setOnlineData_Promise and removeLocalData_Promise promises and this results in getting them to execute in the correct order, however they still execute asynchronously.

Please help if you can!


Here is the code for my other functions FYI:

  setData_Promise(storageKey,dataArray) {
    return new Promise((resolve, reject) => {
      this.storage.set(storageKey, dataArray).then(() => {
        resolve(true);
      }, (err) => {reject(err);});
    });
  }

  getData_Promise(storageKey){
    return new Promise((resolve, reject) => {
      this.storage.get(storageKey).then((data) => {
        if(data){
          resolve(data);
        } else {
          reject(false);
        }
      }, (err) => {reject(err);});
    });
  }
  
  setOnlineData_Promise(localDataObject){

    //Retrieve & return online data
    return new Promise((resolve, reject) => {
      this.getData_Promise('online-db').then((onlineData: any) => {
        if(onlineData){
          //Push localDataObject onto online data
          onlineData.push(localDataObject);
          //Write onlineData to online data
          this.setData_Promise('online-db',onlineData).then((result) => {
            resolve(localDataObject);
          }, (err) => {reject(err);});
          //Return online data
          resolve(onlineData);
        } else {
          //Write localDataObject to online data
          this.setData_Promise('online-db',localDataObject).then((result) => {
            resolve(localDataObject);
          }, (err) => {reject(err);});
        }
      }, (err) => {reject(err);});
    });
  }
  
  removeLocalData_Promise(localDataID){

    return new Promise((resolve, reject) => {
      this.getData_Promise('local-db').then((localData) => {
        if(localData){
          //Delete item from data set
          localData = this.removeByKey_Function(localData, 'id', localDataID);
          //Write returned data set to storage
          this.setData_Promise('local-db',localData).then((result) => {
            resolve(localData);
          }, (err) => {reject(err);});
        } else {
          resolve(false);
        }
      });
    });
  }
  
  removeByKey_Function(array, key, value) {
    for (var i = 0; i < array.length; i++) {
      if(array[i][key]==value){
        array.splice(i,1);
        break;
      }
    }
    return array;
  }

#2

You could check this: http://www.tivix.com/blog/making-promises-in-a-synchronous-manner/

It’s about “generator functions” which return iterators to allow, for example, run Promises in a sync manner.

But they are defined for ECMAScript 6. To use them on ECMAScript 5 (which I suppose you are using) then can try to use this transpiler: https://github.com/facebook/regenerator

In theory this should work. But I haven’t tested it myself.


#3

@UrbanCondor You can do something like:

uploadLocalData_Promise(): Promise<any> {
	return this.getData_Promise('local-db').then(
		localDataArray => this.uploadDataArray(localDataArray) 
	);
}

uploadDataArray(localDataArray: Array<any>): Promise<any> {
	let promiseChain: Promise<any> = Promise.resolve();
	localDataArray = localDataArray || [];

	localDataArray.forEach(localDataObject => {
		promiseChain = promiseChain.then(
			() => this.uploadDataObject(localDataObject)
		);
	});

	return promiseChain;
}

uploadDataObject(localDataObject: any): Promise<any> {
	return this.setOnlineData_Promise(localDataObject).then(() => {
		console.log("Data record #" + localDataObject['id'] + ": Online record created");
		
		return this.removeLocalData_Promise(localDataObject['id']).then(() => {
			console.log("Data record #" + localDataObject['id'] + ": Local record removed");
		});
	}); 
} 

I separated in 3 methods to make the code more readable and easier to understand:

uploadDataObject receives the object that will be uploaded to the online DB and removes it from the local DB. Pay attention that the outer promise, after called successfully (the then is called), receives the inner promise as a result, so this chain will end after the 2 steps are done in the correct order (if any is rejected, it will return a rejected promise).

uploadDataArray do the magic. It creates a promise that does nothing and iterates the array, chaining the new promise generated by uploadDataObject. You should pay attention to 2 things:

1) The function () => this.uploadDataObject(localDataObject) inside the then in the iteration will only be called after the chain reaches this point, so each upload will be sequential, and if one fails (the promise is rejected), the subsequent ones won’t be called (this is what you want, if I understood correctly).

2) To make the uploadDataArray returned promise ends after all uploads are done, the chained promise is changed to the new chained promise created at each iteration (see promiseChain = promiseChain.then(() => ...)).

uploadLocalData_Promise is the method that gets the objects array, uploads each object by calling uploadDataArray and returns a promise that ends after all uploads are done.


#4

Are the various data items independent of one another? If so, isn’t it really the case that all you’re concerned about is “don’t remove anything from local storage until it’s uploaded successfully”? In that case, can’t you let each data item be processed potentially in parallel? I think that would be an improvement.


#5

@lucasbasquerotto - this code you provided appears to be working as I would like it to - thank you so much for taking the time to create such a detailed response. You understand correctly, in this particular instance I would like the workflow to stop if one promise fails.

@rapropos - I totally agree with your comment regarding proceeding in parallell. In this particular instance I would prefer everything to flow synchronously but executing in parallell was going to be my backup plan if I couldn’t achieve what I wanted. I agree that in most instances processing in parallell would be considered an improvement.