The centerpiece of that feature is a green dot. A tiny circle next to a profile picture that means: this person is here, right now.

I hit offline detection bugs that kept users "online" 40 seconds after they'd closed the app. I watched my server silently stop accepting new connections at 300 users because I'd wired the whole system into one binary. I found an article on systemdesign.one, spent a weekend with ChatGPT breaking it down, rebuilt the whole thing, named a goroutine the Reaper, and eventually stress-tested it to 51,322 concurrent connections on my laptop.

This is the full story. Including the parts that hurt.

Where I Started: gRPC Streaming and the First Problem

The transport choice was gRPC streaming from the start. Clients open a long-lived stream to the server and subscribe to presence updates for a list of user IDs. When anyone in that list changes status, the server pushes an event down the stream. One persistent connection per client.

That part was correct.

The first design question: how does the server know a user is online? Heartbeats. When a client connects, the gateway starts a goroutine that calls RecordHeartbeat on the presence service every 15 seconds with the user's ID and a timestamp. The presence service writes that timestamp to Redis. As long as heartbeats arrive, the user is online.

Online detection worked immediately. Offline detection is where things started breaking.

Problem 1: Nobody Was Going Offline

Attempt 1: Redis Keyspace Notifications for Offline Detection

I googled "redis presence detection" and found keyspace notifications. Set a TTL key per user. When it expires, Redis fires an event. Subscribe to those events. Offline detection, free, no extra goroutine, elegant.

I genuinely felt smart for about four hours.

Then I read that keyspace notifications fire for every key event in your entire Redis instance. Not just yours. Every SET, every DEL, every expiry across all your keys. And Redis doesn't expire keys in real time -- it uses lazy expiration or a periodic sweep. Users were showing offline 30 to 40 seconds after reconnecting. The expired key sat un-swept. The new heartbeat key sat ignored. Two conflicting states, zero resolution, one confused frontend.

Why it failed:

• Key expiration in Redis is not deterministic [1]

• Keyspace notifications fire on every key event cluster-wide -- CPU cost scales with total Redis write volume, not your user count

• You're trusting Redis's garbage collector to be your presence engine. Bold.

Problem 2: One Binary, All the Problems

Attempt 2: One Binary to Rule Them All

Keyspace notifications were out. I moved to a heartbeat-based model -- the client sends a signal every 15 seconds, the server stores it in Redis with a timestamp, and I'd figure out offline detection separately. That was the right call.

What came next was not.

I had gRPC streaming. I had heartbeats. I put everything into one binary.

Every stream connection held its own Redis Pub/Sub subscription. Every heartbeat triggered a direct database call from inside the stream handler. Around 300 concurrent streams, the default Redis client pool (10 connections) was fully saturated. New connections blocked waiting for a free slot. PostgreSQL followed. The server didn't crash dramatically. It just... stopped accepting new connections. Quietly. Like it gave up.

Why it failed:

• One Redis Pub/Sub subscription per client connection

• One database call per heartbeat, per stream goroutine

• Default connection pools hit the ceiling before you hit 400 users

The problem was architectural. Stateful connection management and stateless business logic do not belong in the same binary. Their connection pools compete. The long-lived streams always win.

The Architecture That Survived

At some point during Attempt 2, I stopped writing code and started reading.

I found the Real-Time Presence Platform System Design article on systemdesign.one [4]. Then I spent a weekend with ChatGPT going through it section by section the delayed-trigger pattern for offline detection, the gateway split, the single shared Pub/Sub subscription. Every piece had a reason. I just hadn't understood the reasons yet.

The pattern: separate the stateful connection layer (the gateway, which holds open streams) from the stateless logic layer (the presence service, which processes heartbeats and publishes events). One Redis Pub/Sub subscription in the gateway fans out to every connected client. The presence service never touches a client connection directly.

I rebuilt the whole thing. Here's what actually works.

The flow:

1. Client opens a gRPC stream to the Gateway with a list of user IDs they want to watch

2. Gateway starts a heartbeat loop, calling the API server's internal RecordHeartbeat RPC every 15 seconds

3. Presence Service writes to a Redis Sorted Set (score = timestamp, member = userID)

4. If it's a new connection, it publishes an online event to Redis Pub/Sub

5. The Reaper goroutine sweeps for expired users every 5 seconds, publishes offline events

6. Gateway's EventsListener picks up events from Redis and fans them out to the right connections

Why One Redis Command Replaced Three Data Structures

You could use a Redis Set for who's online. A Hash for last-seen timestamps. A separate TTL key per user for expiration. Three data structures, three commands, three places to get out of sync.

A Sorted Set does all of it in one structure.

ZADD presence:active_users <unix_timestamp> <user_id>

ZADD is idempotent: one command handles both insert and update. No SET+EXPIRE dance, no branching.

To find and clean expired users (offline for more than 30 seconds):

ZRANGEBYSCORE presence:active_users -inf <now - 30>
ZREM presence:active_users <expired_user_ids...>

All O(log N). At 100k users, that's 17 comparisons. Redis barely notices.

The alternative was using key-per-user with TTL expiry and keyspace notifications. We tried that. It's Attempt 2. It's in the graveyard.

How the Heartbeat Works

Every 15 seconds, the gateway lies to the presence service. It tells it the user is still online. The presence service believes it, updates the timestamp, and if it's a new connection, shouts "online" to Redis Pub/Sub. If the lies stop coming, the Reaper sweeps up.

// internals/presence/repository.go

const (
    presenceZSetKey = "presence:active_users"
    ttlSeconds      = 30
)

func (r *Repository) RecordHeartbeat(ctx context.Context, userID string) (bool, error) {
    now := float64(time.Now().Unix())
    cutoff := float64(time.Now().Unix() - ttlSeconds)

    score, err := r.redisClient.ZScore(ctx, presenceZSetKey, userID).Result()
    var isNewConnection bool

    if err == redis.Nil {
        // User not in the set at all. Fresh connection.
        isNewConnection = true
    } else if err == nil && score < cutoff {
        // User exists but their last heartbeat expired. They're back.
        isNewConnection = true
    } else if err != nil {
        return false, err
    }

    // Update (or add) the user's timestamp
    err = r.redisClient.ZAdd(ctx, presenceZSetKey, redis.Z{
        Score:  now,
        Member: userID,
    }).Err()

    return isNewConnection, err
}

Source: internals/presence/repository.go

The isNewConnection flag matters. If a user is already online and sends another heartbeat, we don't publish a redundant online event. Only genuine state transitions get broadcast:

// internals/presence/service.go

func (s *Service) RecordHeartbeat(ctx context.Context, req *pb.HeartBeatRequest) (*emptypb.Empty, error) {
    isNewConnection, err := s.repo.RecordHeartbeat(ctx, req.UserId)
    if err != nil {
        log.Printf("Failed to record heartbeat for user %s: %v", req.UserId, err)
        return &emptypb.Empty{}, err
    }

    if isNewConnection {
        event := gateway.PresenceEvent{
            UserID:   req.UserId,
            Status:   "online",
            LastSeen: time.Now().Unix(),
        }
        payload, err := json.Marshal(event)
        if err == nil {
            s.redisClient.Publish(ctx, "system:presence_events", payload)
        }
    }

    return &emptypb.Empty{}, nil
}

Source: internals/presence/service.go

The gateway side. keepAliveLoop runs as a goroutine for the lifetime of the stream:

// internals/gateway/handler.go

func (s *Server) keepAliveLoop(ctx context.Context, userID string) {
    ticker := time.NewTicker(15 * time.Second)
    defer ticker.Stop()

    s.sendHeartbeat(userID) // fire immediately on connect

    for {
        select {
        case <-ctx.Done():
            return
        case <-ticker.C:
            s.sendHeartbeat(userID)
        }
    }
}

Source: internals/gateway/handler.go

Heartbeat interval is 15 seconds. TTL is 30 seconds. That gives two missed heartbeats before the user is considered offline. One missed heartbeat from a flaky mobile connection doesn't trigger a false offline event.

How Offline Detection Works: The Reaper

After heartbeats were working, I had a new problem. Users who closed the app stayed "online" on everyone else's screen. 30 seconds. 40 seconds. Sometimes longer.

There was no disconnect event. No callback. When the gRPC stream closed, the gateway's heartbeat goroutine exited. That was it. Nobody told the presence service. Nobody published an offline event. The user's timestamp just sat in the sorted set, aging, while their green dot stayed green.

I needed something to notice the silence.

I wrote a goroutine that runs every 5 seconds, queries Redis for users whose last heartbeat is older than 30 seconds, removes them from the sorted set, and publishes an offline event for each one. I called it the Reaper.

// internals/presence/reaper.go

func (r *Reaper) Start(ctx context.Context) {
    ticker := time.NewTicker(5 * time.Second)
    defer ticker.Stop()

    for {
        select {
        case <-ctx.Done():
            return
        case <-ticker.C:
            r.sweep(ctx)
        }
    }
}

func (r *Reaper) sweep(ctx context.Context) {
    sweepCtx, cancel := context.WithTimeout(ctx, 4*time.Second)
    defer cancel()

    expiredUsers, err := r.repo.GetAndRemoveExpiredUsers(sweepCtx)
    if err != nil {
        log.Printf("Reaper failed to fetch expired users: %v", err)
        return
    }

    if len(expiredUsers) > 0 {
        now := time.Now().Unix()
        for _, userID := range expiredUsers {
            event := gateway.PresenceEvent{
                UserID:   userID,
                Status:   "offline",
                LastSeen: now,
            }
            payload, err := json.Marshal(event)
            if err == nil {
                r.redisClient.Publish(sweepCtx, "system:presence_events", payload)
            }
        }
    }
}

Source: internals/presence/reaper.go

The sorted set query and removal:

// internals/presence/repository.go

func (r *Repository) GetAndRemoveExpiredUsers(ctx context.Context) ([]string, error) {
    cutoff := fmt.Sprintf("%f", float64(time.Now().Unix()-ttlSeconds))

    // fetch all users whose last heartbeat score is older than 30s
    expiredUsers, _ := r.redisClient.ZRangeArgs(ctx, redis.ZRangeArgs{
        Key:     presenceZSetKey,
        ByScore: true,
        Start:   "-inf",
        Stop:    cutoff,
    }).Result()

    if len(expiredUsers) > 0 {
        // convert to []interface{} for ZRem
        members := make([]interface{}, len(expiredUsers))
        for i, u := range expiredUsers { members[i] = u }
        r.redisClient.ZRem(ctx, presenceZSetKey, members...)
    }

    return expiredUsers, nil
    // full implementation: https://github.com/dis70rt/bluppi-backend/blob/main/internals/presence/repository.go
}

Source: internals/presence/repository.go

The 4-second context timeout on sweep is intentional. If Redis is unresponsive, the Reaper doesn't hang forever and block subsequent sweeps. It logs the failure and tries again in 5 seconds. LinkedIn's presence platform uses a similar delayed-trigger approach for offline detection [3].

The Gateway: Multi-Device Connections and Targeted Fanout

The ConnectionManager uses a nested map: userId -> connectionId -> Connection. Each device gets a UUID. Disconnecting one device only cleans up that specific connection.

// internals/gateway/manager.go

type Connection struct {
    ID                 string
    UserID             string
    Chan               chan PresenceEvent
    SubscribedToTarget []string // which users this connection is watching
}

type ConnectionManager struct {
    mu sync.RWMutex
    // userId -> connectionId -> Connection
    userConnections map[string]map[string]*Connection
    // targetUserId -> set of connectionIds watching them
    targetWatchers map[string]map[string]struct{}
    // connectionId -> Connection reference (for quick lookup during push)
    connById map[string]*Connection
}

Source: internals/gateway/manager.go

Three maps. Looks like overkill. It's not.

• userConnections: "which connections does this user have?" Needed for cleanup.

• targetWatchers: "who's watching this user?" Needed for fanout. This is the inverted index.

• connById: "give me the connection object by ID." Needed for pushing events.

When a presence event arrives for User B, the PushEvent method looks up User B in targetWatchers, gets the set of connection IDs watching them, resolves each to a Connection via connById, and pushes the event to its channel:

// internals/gateway/manager.go

func (cm *ConnectionManager) PushEvent(targetUserID string, event PresenceEvent) {
    cm.mu.RLock()
    defer cm.mu.RUnlock()

    watchers, exists := cm.targetWatchers[targetUserID]
    if !exists {
        return
    }

    for connID := range watchers {
        if conn, ok := cm.connById[connID]; ok {
            select {
            case conn.Chan <- event:
            default: // Drop event if channel is full to avoid blocking
            }
        }
    }
}

Source: internals/gateway/manager.go

That select { default: } is load-bearing. Without it, a slow consumer blocks the entire fanout. With it, a full channel just drops the event. The user misses one status update and gets the next one 15 seconds later. Better than a deadlock that freezes the entire gateway.

The Redis Pub/Sub listener ties it together. One goroutine, one subscription, fans out to all connections:

// internals/gateway/listener.go

func (el *EventsListener) Start(ctx context.Context) {
    pubsub := el.redisClient.Subscribe(ctx, "system:presence_events")
    defer pubsub.Close()

    for {
        select {
        case <-ctx.Done():
            return
        case msg := <-pubsub.Channel():
            var event PresenceEvent
            if err := json.Unmarshal([]byte(msg.Payload), &event); err != nil {
                log.Printf("Failed to unmarshal presence event: %v", err)
                continue
            }
            el.connManager.PushEvent(event.UserID, event)
        }
    }
}

Source: internals/gateway/listener.go

Goroutines as Actors: How Go Solved the Mutex Problem

Here's the part nobody explains when they show you a sync.RWMutex slapped on a map and call it a day.

The ConnectionManager has a mutex. It protects the three index maps: registration, deregistration, lookups. That's the right scope for a lock.

But event delivery has no lock at all. PushEvent holds only an RLock to read the watcher index, then sends to a channel. It never waits. It never coordinates. Each connection processes its own events in complete isolation.

This is the Actor Model [8], implemented with zero framework, zero library, zero ceremony. In Go, a goroutine plus a channel is an actor. The channel is the mailbox. The goroutine is the event loop. Messages never share memory across actors.

Look at SubscribePresence:

// internals/gateway/handler.go

func (s *Server) SubscribePresence(req *pb.SubscribeRequest, stream pb.PresenceGateway_SubscribePresenceServer) error {
    ctx := stream.Context()

    userID, err := middlewares.GetUserID(ctx)
    if err != nil {
        return err
    }

    conn, connID := s.connManager.AddConnection(userID, req.TargetUserIds)
    defer s.connManager.RemoveConnection(userID, connID)

    go s.keepAliveLoop(stream.Context(), userID)

    for {
        select {
        case <-stream.Context().Done():
            return stream.Context().Err()

        case event, ok := <-conn.Chan: // the actor receiving its mailbox message
            if !ok {
                // channel closed by RemoveConnection: clean shutdown signal,
                // no extra sync primitive needed
                return nil
            }
            grpcMessage := &pb.PresenceUpdate{
                UserId:   event.UserID,
                Status:   event.Status,
                LastSeen: timestamppb.New(time.Unix(event.LastSeen, 0)),
            }
            if err := stream.Send(grpcMessage); err != nil {
                return err
            }
        }
    }
}

Source: internals/gateway/handler.go

Each call to SubscribePresence is its own actor instance. It has one goroutine, one channel (conn.Chan, capacity 20), one job: drain the mailbox and stream events to the client. It shares no state with any other connection's event loop.

The alternative is a shared event queue with a mutex on every dequeue. At 50k concurrent connections, every event delivery serializes through one lock. Under that contention, sync.Mutex triggers O(N) goroutine wake-ups per unlock. The scheduler spends more time arbitrating lock ownership than your code spends doing actual work.

With the actor approach: PushEvent does one non-blocking channel send and moves on. No coordination. No waiting. The worst case is a dropped event when a consumer's buffer is full -- which is a better outcome than a deadlocked gateway serving nobody.

The only mutex in the entire system is on the maps. Not on message delivery. That's the point.

The Two-Binary Split: API Server vs. Gateway Server

The API server (cmd/api/main.go, port 50051) handles business logic: users, music, rooms, parties, and the internal RecordHeartbeat RPC. Stateless. Horizontally scalable. Talks to PostgreSQL, Memgraph, Redis, Solr.

The Gateway server (cmd/gateway/main.go, port 50050) handles long-lived gRPC streams. Stateful by nature (it holds open connections). Talks to Redis Pub/Sub and the API server over internal gRPC. (Note: you can use WebSockets instead of gRPC streams here; the core principles of connection management and fanout remain exactly the same.)

The gateway authenticates clients via Firebase Auth through a gRPC stream interceptor, then calls the API server with the user ID attached as gRPC metadata. The API server never sees raw client connections for presence. It processes heartbeats and that's it.

Load Test: 51,322 Users, Zero Errors

I wrote a custom load test tool in Go (cmd/loadtest/presence/main.go) that spins up concurrent gRPC streams. Each simulated user subscribes to their own events, connects, receives their online event, and holds the stream open.

Important caveat: this is a connection-overhead test, not a fanout test. Each user watches only themselves. In production, one popular user might have thousands of watchers -- that fanout path has different latency characteristics. The numbers below measure how many concurrent streams the gateway holds open without errors, not worst-case fanout latency.

Results from a single laptop (i5-13500HX, 15 GiB RAM):

*Target was 100k. Stopped at 51,322 because the load generator ran out of RAM. The backend was fine. My laptop was not.

Zero errors across all runs. p99 latency at 51k users: 15.28 ms. The backend didn't break. The load generator did.

For context, LinkedIn targets sub-second presence delivery for their 700M+ user base [3]. Our p99 at 51k is 15ms. Different scale, sure, but the architecture patterns are the same.

Known Limitations

I'm still working on some of these. Listing them anyway because honesty is more useful than a blog post that pretends everything is solved.

• Single-node gateway. If the gateway goes down, all streams drop. No cross-node fanout yet. Fixing this requires a dispatcher layer similar to LinkedIn's architecture [3] or consistent hashing across gateway instances.

• Reaper sweep interval. The 5-second tick means offline detection can lag up to 35 seconds (30s TTL + 5s sweep). For a music app's friends activity, this is fine. For an ICU patient monitoring system, this would be a resume-generating event.

• No cross-data center replication. Single region. Adding geo-distribution would need CRDT-based replication on the Redis layer, similar to what the systemdesign.one article describes [4].

• ulimit capped the load test. The default 1024 open file limit on the load generator machine was the actual bottleneck. Raising it and using distributed load generators would push past 100k.

Closing

This took months. For a green dot.

The full source is on GitHub: dis70rt/bluppi-backend. The presence system lives in internals/presence/ and internals/gateway/. Go read it, break it, tell me what's wrong with it.

Two things I wish I knew at the start:

If you're reaching for Redis keyspace notifications for offline detection -- don't. Write a Reaper. It's 66 lines of Go and it does exactly what you need with zero surprises.

If you're about to let every client hold a Redis Pub/Sub subscription from inside your API server -- also don't. Split the gateway out. The connection pool you save might be your own.

And if you're a student building something that "sounds simple" and it starts taking way longer than expected: that's normal. That's just what real systems feel like when you stop using the whiteboard version.

References

1. Redis documentation, "Redis keyspace notifications." redis.io/docs

2. gRPC documentation, "Keepalive." grpc.io/docs

3. Gupta & Lay, "Now You See Me, Now You Don't: LinkedIn's Real-Time Presence Platform" (2018). engineering.linkedin.com

4. Kim, "Real-Time Presence Platform System Design" (2023). systemdesign.one

5. Barber, "Building Real-Time Infrastructure at Facebook" (2017). SREcon / USENIX. youtube.com

6. PubNub, "Is Anyone Home? An Intro to Presence Webhooks" (2020). pubnub.com

7. Redis documentation, "Sorted Sets." redis.io/docs

If you're here, you probably already know that Firebase is great for building apps with real-time data. However, many developers run into a big issue—Firebase costs can quickly get out of hand due to too many read operations.

In 2018, a startup in Colombia faced this exact problem when they scaled up to 2 million daily active users (DAUs). A small, but costly mistake in their code ended up leaving them with a $30,356.56 bill from Google Clouds in just 72 hours. Why? Because that tiny error caused 2 million users to each trigger 16,000 document reads, leading to over 40 billion requests to Firestore in less than 48 hours. Read the full article

You can avoid such excessive costs by optimizing your reads. In this blog, we'll dive into specific strategies to stop paying for every single read without sacrificing app performance.

Understanding Firebase Read Costs

Firebase's pricing model is simple. In Firestore, you are charged based on the number of documents you read. Each time a document is read from the database—whether through a query, real-time listener, or a simple fetch—it counts as a read.

Check out the Firestore Pricing here.

Before jumping into solutions, it’s important to understand where things can go wrong.

• Fetching unnecessary data: For instance, if you’re retrieving large collections when you only need a few documents, you’re paying for extra reads.

• Inefficient queries: Poor query structuring can result in pulling unnecessary data or performing repeated reads.

• Overuse of real-time syncing: Real-time listeners are great, but they can result in multiple reads as they sync data even when it’s not needed immediately.

Strategies to Reduce Firebase Read Costs

Optimize Your Data Structure

A well-organized database can greatly cut down the number of reads. If your Firestore database isn't structured efficiently, it can cause you to fetch more data than needed, leading to extra read costs.

Choosing when to use nested data vs flattened data is crucial for reducing unnecessary reads in Firestore.

Example: Instead of storing all user posts in one collection, divide them into smaller subcollections by category or user ID.

Query Optimization

Firestore is a NoSQL database, and it's important to use its indexing features. Writing efficient queries can help lower the number of documents read.

Firestore automatically indexes each document by its document ID. But for complex queries, like filtering on several fields, you might need to create composite indexes.

Example: If you want to query users by both age and city, you'll need a composite index for age and city.

Filtering documents with where() clauses retrieves only the necessary data.

final QuerySnapshot querySnapshot = await FirebaseFirestore.instance
      .collection('users')
      .where('age', isGreaterThanOrEqualTo: 18)
      .where('city', isEqualTo: 'New York')
      .get();

Make sure to combine filters that make sense together to reduce the number of reads.

Implement Pagination Using limit() for Efficient Data Retrieval

final int pageSize = 10;
DocumentSnapshot? lastVisible;

Future<void> getPosts() async {
  final query = FirebaseFirestore.instance
      .collection('posts')
      .orderBy('createdAt')
      .limit(pageSize)
      .startAfterDocument(lastVisible!);
  
  final querySnapshot = await query.get();

  querySnapshot.docs.forEach((doc) => print(doc.data()));
  lastVisible = querySnapshot.docs.isNotEmpty ? querySnapshot.docs.last : null;
}

Limiting documents can significantly reduce costs and improve performance, especially when dealing with large datasets.

Avoid Unnecessary Real-time Listeners

Real-time listeners are powerful but expensive if used without careful consideration. In many cases, you don't need real-time updates for every piece of data.

Manual Refreshing: Instead of relying on real-time updates, users can manually trigger a refresh to load the latest data.

In this blog, we will look at advanced caching methods that can greatly reduce your Firestore costs. These strategies are more than just basic offline caching and can save you a lot of money while improving your app's performance.

Advanced Strategies to Reduce Firebase Costs

You can use SharedPreferences to store small pieces of data that are frequently accessed, like user info or settings. This works well for data that doesn’t change much. However, SharedPreferences is only suitable for small amounts of data. If you store too much, it can slow down your app, use more memory, and hit storage limits.

import 'package:shared_preferences/shared_preferences.dart';

Future<void> saveUserInfo(String name, int age, String email) async {
  final SharedPreferences prefs = await SharedPreferences.getInstance();
  await prefs.setString('userName', name);
  await prefs.setInt('userAge', age);
  await prefs.setString('userEmail', email);
}

For example, you can store user information in SharedPreferences as small chunks of data that are repeatedly used throughout your application.

Use Provider for efficient state management to reduce Firestore reads.

Without proper state management, your app might request the same data from Firestore multiple times, increasing the number of reads unnecessarily. By using Provider, once data is fetched from Firestore, it can be stored and accessed efficiently throughout the app without additional Firestore calls.

import 'package:flutter/material.dart';
import 'package:cloud_firestore/cloud_firestore.dart';

class UserProvider extends ChangeNotifier {
  Map<String, dynamic>? _userData;
  Map<String, dynamic>? get userData => _userData;

  Future<void> fetchUserData(String userId) async {
    if (_userData != null) return; // Skip fetching if data is already loaded

    try {
      final doc = await FirebaseFirestore.instance.collection('users').doc(userId).get();
      _userData = doc.data();
    } catch (e) {
      print('Error fetching user data: $e');
    } 
      
    notifyListeners();
  }
}

Don’t forget to add the Provider class to the main.dart file and then use it in the UI. This ensures the data is fetched only once when the app starts and is shared efficiently across all UI components, screens, and pages.

If you know that the data is not going to change frequently, you can save it on the client side, essentially caching the data. This can be achieved through various methods, such as using Hive, SQLite, or path_provider, each serving a different purpose.

Server-Side Optimization for Cost Reduction

If all the users are requesting the same data from your Firebase database, it can unnecessarily increase your reads. To avoid this, you can use the Firebase Admin SDK to create bundles, request the data only once, store it in those bundles, and send the bundles to the clients. This will reduce Firebase reads and improve your app's performance, as it only needs to request the data once.

from google.cloud import firestore
from google.cloud.firestore_bundle import FirestoreBundle

db = firestore.Client()
bundle = FirestoreBundle("bundle-name")

for user in db.collection("users").stream():
    bundle.add_document(user.get())
    
    for post in db.collection("posts").where("user_id", "==", user.id).stream():
        bundle.add_document(post.get())

bundle_buffer = bundle.build()

with open("cacheBundle.bin", "wb") as file:
    file.wrtie(bundle_buffer)

The above code snippet might not be entirely correct as I forgot the exact syntax, but it was something similar to this. Check Docs

Once you've created the Firestore bundle and serialized it, you can save the bundle as a file (e.g., cacheBundle.bin) and send it to all of your clients via HTTP or upload it to Firebase Cloud Storage. This way, clients can access the cached data from the bundle without repeatedly making requests to Firebase Database.

Conclusion

Optimizing Firebase reads can help you save costs and improve app performance. By structuring your data efficiently, optimizing queries, minimizing real-time listeners, and using caching strategies, you can reduce unnecessary reads without sacrificing performance.

Feel free to share your own methods or any corrections in the comments below. Happy coding, and may your Firebase costs stay low while your app thrives!