App with large data


#1

Hi, I need to develop an app with more data (read and search). There are about 100000 record. How can implement it? What is best storing approch? There is some limit with ng-repeat? Thanks


#2

You can use virtual scrolling. Virtual Scroll displays a virtual, “infinite” list. An array of records is passed to the virtual scroll containing the data to create templates for. The template created for each record, referred to as a cell, can consist of items, headers, and footers.

Read Here :


#3

Thanks. But the best Solution is use array of element in a service or an sql lite db ti store all data? Thanks


#4

The question is too vague for any chance of a helpful answer.

100,000 records says little about the volume of data. 100k byte-sized records? …MB sized records? do you have strict concerns for referential integrity (eg: do you have concerns over broken links, because that’s a LOT of local processing you seem to need to do)? or if this is for NoSQL, what depths do your documents (records) reach?

what’s the concern that makes you want to force a phone to do data processing?? (this is a high bar to reach) and why would that concern outweigh the massive performance hits your users are about to have? I’m assuming this is for the iOS and Android platforms. Is that the case?

I’m sure a lot of people would like to help and weigh in - it seems interesting - but you need to supply the full problem. hypotheticals don’t bring actionable answers.


#5

Hi,
I neet storage same catalog record. (i.e. SKU | DESCRIPTION | PRICE | LINK)
There are same catalog with about 100000 record everyone.
Size of txt file is about 2Mb (1 record for line)
In my application i need to store about 10 ten of this data file.
I need show in list (but with previous answer that is ok), and i need to search it.
I need deploy app in android and ios platform.

Thanks


#6

Ten different 2MB files? Is that right?

Why will the files need to be loaded locally? Why can’t they be stored on your server? Are the 100k entries all customized to each individual user?

What I’m getting at is that your problem -so far- seem to lie in the architecture design, and this will dictate your code implementation.


#7

Let me add this very common issue to your question. I hope this one issue alone will make you see that you should spend time looking into the architecture before you get into the code:

What are you going to do when just 1 of the 100,000 items needs its description updated? Push an app update to all of your users? None of the app code needed changing.

What will you do when you want to add the 100,001st item?

Here’s a page to start reading… http://www.drdobbs.com/web-development/building-scalable-web-architecture-and-d/240142422


#8

Hi, because app need to work offline.
I think to add this embedded in file or into sql lite table.
No upadate or future update are requested.


#9

Hi,
My actual solution provide about 100000 static record. No update is needed. I try to add as array, but when I call ng-repeat to display, service give me an error - seem that I can’t use ng-repeat with all record. Error return me that record limit is reached. There is a limit to record that will be showing ? Thanks


#10

Array of 100K length … ng-repeat with 100K nodes … both are bad ideas … what you can try is store your 100K records in some persistent storage (preferably SQLite ) then implement some sort of infinite scrolling design which will load a small set of data at any point of time or pagination … You can asynchronously request DB to fetch records … this will be like a server call just it would work in offline as you are calling a local data store