I need to load a couple thousand records of user data (user contacts in a contact-management system, to be precise) from a REST service and run a seach on them. Unfortunately, the REST service doesn't offer a search which meets my needs, so I'm reduced to just loading a bunch of data and searching through it myself. Loading the records is time-consuming, so I only want to do it once for each user.
Obviously this data needs to be cached. Unfortunately, server-side caching is not an option. My client runs apps on multiple servers, and there's no way to predict which server a given request will land on.
As an alternative, proxy the REST Service in question, and create your own search on a server that you then control. You could do this with pretty quickly and easily with Python + Django + XML Models. No doubt there are equally simple ways to do this with whatever your preferred dev language is. (In re-reading, I see that you can't do server-side caching which may make this point moot).
You can manage tens of thousands of records safely in the browser. I'm running search & sorting benchmarks with jOrder (http://github.com/danstocker/jorder) on such datasets with no problem.
I would look at a distributed server side cache. If you keep the data in the browser, as system grows you will have to increase the browser cache lifetime to keep traffic down.
©2020 All rights reserved.