Allow for custom implementations of QueryCache (limit cache size)

[damianon]

The use case where I'm currently trying to use tanstack-query has one rather peculiar requirement: Limit the size of the cache.

This can either be done by size (probably more complicated) or by item count (more arbitrary, but easier). I've tried to find a way to configure this, but couldn't find any. The closest was implementing (or extending) the QueryClient, but unfortunately to extend it, the this.query property is private, so manually limiting it in any of the ways described isn't possible.

Btw: using time (cacheLimit) isn't an option, because users can trigger quite a few queries in a short amount of time.

Are there any suggestions on how this can be achieved, or can tanstack-query make it easier to create a custom cache implementation (would also make it possible to use different "backends" for the cache)

[TkDodo]

Yeah I was thinking about this since swr has this (or will have it in v2). All we would need to do is expose the possibility to overwrite the internal Object / Map with something that matches the interface. So you could do:

new QueryClient({
  queryCache: new QueryCache({
    cache: { get, set, delete } // this doesn't exist yet, don't try it.
  })
})

so then you could use your own implementation and do whatever you want with it... I just don't know how you would like to enforce the limit? Like, what happens if set is called but there is no space? You likely can't just remove something else, because it might still have active observers, which could then try to write again to the cache. Can you elaborate on the use-case of limiting the cache size?

[damianon]

Basically the application can be quite long-running, but we still want to maintain the cache as long as possible. However, a user would trigger a lot of queries in a very short amount of time and the queries can end up taking up a lot of space (the application is unfortunately not the newest).

But we've found another approach now: using an instance of QueryCache we check it using setInterval and remove inactive old cache entries if a certain threshold is passed

[TkDodo]

Just to be on the same page: queries that are unused can be removed immediately when they become unused by setting cacheTime: 0. Removing active queries is not a good idea imo.

[damianon]

Yes, but as mentioned: cache should be retained for as long as possible (even if stale), and we only delete inactive queries, when the cache grows too big.

[devqazi]

@damianon Is there a documented limit for "too big"? I stumbled on this thread trying to find how much data react-query holds before refusing to store more. Is there a way to force a limit say: "Do not store more than 200 unique respones per queryFn"?

[Yangziyu]

已收到你的邮件。

[Que-tin]

@TkDodo Just stumbled across this one, are there any news on this?

This could probably break a lot of stuff when not done properly, but I feel like this could help a lot. Especially useful for 3rd party tools.

There would need to be proper guidlines or smth. similar to make sure people also implement the notify stuff so it won't break other tooling. Otherwise it would probably lead to confusion and issues on this repo if some 3rd party tool implements some weird stuff there.

[DanielOrtel]

Hey, @TkDodo I see this is added to the V5 roadmap in #4252, though V5 is already here, and I couldn't find anything related to it.

I don't exactly know what's in scope for the manual caching options, but just to add to the use cases, I have a very specific one for this as well: my backend is written on a concept of Resources, and tailor made so that frontend can consume it as efficiently as possible:

To illustrate, queries for a list of items always return in the following format:

{
  type: resource-name
  list: Array<ids> // of primary type
  data: Record<id, primary-resource-data
  included: Record<resource-name, Record<id, included-resource-data>> // when clients request related resources as part of the request
}

With get requests(and responses for mutations) following a similar pattern:

{
  data: Record<primary-resource-data>
  included: Record<included-resource-name, Record<ids, included-resource-data>>
}

What this essentially means, is that the frontend can use a very specific caching strategy:

• resource + id -> this will always be unique, so the data received from list and get requests can safely just add the data and included fields to the cache. So if there is a page that is accessing that resource + id, it can just fetch from the resource + id cache, instead of doing another request to the server, when the previous list request has already fetched it.
• list requests can be cached by page + filter + order parameters exclusively, since with list requests' primary result is the list attribute, so that is the only thing we need to cache, the rest of the resources are cached by name and id separately.

This cache setup would allow a lot of manual fiddling to be automated, such as query invalidation from mutations, automatic updates after mutations, etc.

Compared to the current cache setup, it would also significantly reduce the cache size: currently every query is cached individually, so we have a lot of duplication between different kind of fetches for separate resources.

[DanielOrtel]

FYI: I'm open to helping with the implementation, if needed

[TkDodo]

we wanted to expose the Map we created internally, and let you create a separate one. The problem is that this isn’t very helpful, because it would still be the same “QueryCache” class instance. The use-cases we wanted to support with this weren’t actually using this, so we dropped it.

[DanielOrtel]

Thanks, I see, so there are no plans currently to support such cases?

[TkDodo]

it can just fetch from the resource + id cache, instead of doing another request to the server

I think you can do that right now by implementing a queryFn that reads from a different cache key