Firebase integration for tracking leaderboard positions

Question

Firebase integration for tracking leaderboard positions

Currently, I have a project where I need to showcase a leaderboard featuring the top 20 users, with those not included still appearing in 21st place alongside their current ranking.

I'm looking for an efficient way to accomplish this task. My database of choice is Cloud Firestore, although I now realize that MongoDB might have been a better option. However, given my progress on the project, I need to stick with Cloud Firestore.

The app is expected to have around 30K users. Is there a method to achieve this without retrieving data for all 30K users?

this.authProvider.afs.collection('profiles', ref => ref.where('status', '==', 1)
        .where('point', '>', 0)
        .orderBy('point', 'desc').limit(20))

The above code successfully fetches the top 20 users, but I am unsure of the best approach to determine the rank of the currently logged-in user if they do not fall within the top 20 positions.

javascript database firebase angularfire2 google-cloud-firestore

Answer 1

Answer №1

Finding the position of a random player in a leaderboard, in a way that is scalable, presents a common challenge when dealing with databases.

Several factors will influence the solution you choose, including:

Total number of players
The speed at which individual players are accumulating scores
The rate at which new scores are being added (number of concurrent players * previous factor)
Score range: Limited or Unlimited
Distribution of scores (uniform, or specific 'hot scores')

Simple Approach

The usual simple method involves counting all players with a higher score, for example

SELECT count(id) FROM players WHERE score > {playerScore}

.

While this approach works for small scales, it becomes slow and resource-intensive as your player base expands, especially in MongoDB and Cloud Firestore.

Cloud Firestore does not support count natively due to scalability issues. You would need to count the returned documents on the client-side or use Cloud Functions for Firebase for server-side aggregation to avoid extra data transfer.

Regular Update

Instead of updating rankings continuously, consider updating them periodically, like once every hour. Take Stack Overflow's ranking system as an example, where rankings are updated daily.

To implement this, you could schedule a function, or schedule App Engine if the process takes more than 540 seconds. This function would create a list of players in a ladder collection with a new rank field. Retrieving the top X players plus the player's rank would then be efficient in O(X) time.

An optimization would be to explicitly store the top X players in a single document, reducing the read operation to just 2 documents, enhancing performance and saving costs.

This method can cater to any number of players and write rates since it operates independently. Adjusting the update frequency may be necessary as your player base grows based on financial considerations. With 30K players per hour, the cost would amount to $0.072 per hour ($1.73 per day), unless optimizations are applied (e.g., excluding players with zero scores).

Inverted Index

In this technique, an inverted index structure is created. It is effective when there is a limited score range much smaller than the total number of players (e.g., scores from 0-999 against 30K players). Alternatively, it could also work for an unbounded score range but with a significantly lower count of unique scores compared to players.

By utilizing a separate collection named 'scores', each unique score has a corresponding document containing a field called player_count.

When a player achieves a new total score, 1-2 writes occur in the scores collection. One write increments player_count for their new score while decrementing the old score if applicable. This method supports both "Current score equals latest score" and "Current score is highest achieved score" ladder styles.

Determining a player's precise rank is as simple as executing something like

SELECT sum(player_count)+1 FROM scores WHERE score > {playerScore}

.

Since Cloud Firestore lacks support for sum(), this computation is done on the client side. The addition of 1 accounts for the player themselves among those above them, determining their rank.

With this approach, reading up to 999 documents (averaging around 500) is required to obtain a player's rank, although this number decreases if scores with no players are removed.

Understanding the write rate of new scores is crucial, as updating an individual score is limited to once every 2 seconds on average. For a fair distribution of scores ranging from 0-999, this translates to supporting approximately 500 new scores per second. Implementing distributed counters for each score can enhance this capability.

* As each score generates two writes, limiting updates to one per 2 seconds
** Assuming an average game duration of 2 minutes, handling 500 new scores per second could accommodate up to 60,000 simultaneous players without distributed counters. Higher volumes are feasible with a "Highest score is current score" mechanism.

Sharded N-ary Tree

Considered the most challenging option, the Sharded N-ary Tree method offers faster and real-time ranking positions for all players. It serves as a read-optimized counterpart to the Inverted Index methodology discussed earlier, which leans towards write optimization.

Refer to the related article on 'Fast and Reliable Ranking in Datastore' for a suitable general strategy. A bounded score is recommended for this approach (although possible with an unbounded range, modifications would be necessary).

I do not recommend this route due to requiring distributed counters for top-level nodes in frequent-update scenarios, potentially offsetting gains in read-time efficiency.

https://i.sstatic.net/V81hf.png

Final Thoughts

By combining different approaches, the leaderboard optimization can be maximized depending on the frequency of player leaderboard views.

Merging 'Inverted Index' with 'Periodic Update' at shorter intervals can provide O(1) ranking accessibility for all players.

If the leaderboard is viewed over 4 times during the 'Periodic Update' timeframe, money can be saved, and the leaderboard will operate faster.

Within each period, such as 5-15 minutes, all documents from scores can be read in descending order. By tracking the total players_count incrementally, scores can be rewritten into a new collection named scores_ranking with an additional players_above field. This field sums up player counts excluding the current score, simplifying rank retrieval.

Obtaining a player's rank now only requires fetching the player's score document from score_ranking -> Their rank corresponds to players_above + 1.

Answer 2

Finding the position of a random player in a leaderboard, in a way that is scalable, presents a common challenge when dealing with databases.

Several factors will influence the solution you choose, including:

Total number of players
The speed at which individual players are accumulating scores
The rate at which new scores are being added (number of concurrent players * previous factor)
Score range: Limited or Unlimited
Distribution of scores (uniform, or specific 'hot scores')

Simple Approach

The usual simple method involves counting all players with a higher score, for example

SELECT count(id) FROM players WHERE score > {playerScore}

.

While this approach works for small scales, it becomes slow and resource-intensive as your player base expands, especially in MongoDB and Cloud Firestore.

Cloud Firestore does not support count natively due to scalability issues. You would need to count the returned documents on the client-side or use Cloud Functions for Firebase for server-side aggregation to avoid extra data transfer.

Regular Update

Instead of updating rankings continuously, consider updating them periodically, like once every hour. Take Stack Overflow's ranking system as an example, where rankings are updated daily.

To implement this, you could schedule a function, or schedule App Engine if the process takes more than 540 seconds. This function would create a list of players in a ladder collection with a new rank field. Retrieving the top X players plus the player's rank would then be efficient in O(X) time.

An optimization would be to explicitly store the top X players in a single document, reducing the read operation to just 2 documents, enhancing performance and saving costs.

This method can cater to any number of players and write rates since it operates independently. Adjusting the update frequency may be necessary as your player base grows based on financial considerations. With 30K players per hour, the cost would amount to $0.072 per hour ($1.73 per day), unless optimizations are applied (e.g., excluding players with zero scores).

Inverted Index

In this technique, an inverted index structure is created. It is effective when there is a limited score range much smaller than the total number of players (e.g., scores from 0-999 against 30K players). Alternatively, it could also work for an unbounded score range but with a significantly lower count of unique scores compared to players.

By utilizing a separate collection named 'scores', each unique score has a corresponding document containing a field called player_count.

When a player achieves a new total score, 1-2 writes occur in the scores collection. One write increments player_count for their new score while decrementing the old score if applicable. This method supports both "Current score equals latest score" and "Current score is highest achieved score" ladder styles.

Determining a player's precise rank is as simple as executing something like

SELECT sum(player_count)+1 FROM scores WHERE score > {playerScore}

.

Since Cloud Firestore lacks support for sum(), this computation is done on the client side. The addition of 1 accounts for the player themselves among those above them, determining their rank.

With this approach, reading up to 999 documents (averaging around 500) is required to obtain a player's rank, although this number decreases if scores with no players are removed.

Understanding the write rate of new scores is crucial, as updating an individual score is limited to once every 2 seconds on average. For a fair distribution of scores ranging from 0-999, this translates to supporting approximately 500 new scores per second. Implementing distributed counters for each score can enhance this capability.

* As each score generates two writes, limiting updates to one per 2 seconds
** Assuming an average game duration of 2 minutes, handling 500 new scores per second could accommodate up to 60,000 simultaneous players without distributed counters. Higher volumes are feasible with a "Highest score is current score" mechanism.

Sharded N-ary Tree

Considered the most challenging option, the Sharded N-ary Tree method offers faster and real-time ranking positions for all players. It serves as a read-optimized counterpart to the Inverted Index methodology discussed earlier, which leans towards write optimization.

Refer to the related article on 'Fast and Reliable Ranking in Datastore' for a suitable general strategy. A bounded score is recommended for this approach (although possible with an unbounded range, modifications would be necessary).

I do not recommend this route due to requiring distributed counters for top-level nodes in frequent-update scenarios, potentially offsetting gains in read-time efficiency.

https://i.sstatic.net/V81hf.png

Final Thoughts

By combining different approaches, the leaderboard optimization can be maximized depending on the frequency of player leaderboard views.

Merging 'Inverted Index' with 'Periodic Update' at shorter intervals can provide O(1) ranking accessibility for all players.

If the leaderboard is viewed over 4 times during the 'Periodic Update' timeframe, money can be saved, and the leaderboard will operate faster.

Within each period, such as 5-15 minutes, all documents from scores can be read in descending order. By tracking the total players_count incrementally, scores can be rewritten into a new collection named scores_ranking with an additional players_above field. This field sums up player counts excluding the current score, simplifying rank retrieval.

Obtaining a player's rank now only requires fetching the player's score document from score_ranking -> Their rank corresponds to players_above + 1.

Answer 3

Answer №2

Another creative solution I have come across involves estimating a user's rank in cases where they are not visible on the leaderboard. In my upcoming online game, I plan to implement this method which could also be applicable in your scenario. After all, does it really matter to the user whether they are ranked 22,882nd or 22,838th?

For example, if the player in 20th place has a score of 250 points and there are a total of 32,000 players, each point below 250 can represent an average drop of 127 places. However, implementing a curve may be beneficial so that as users approach the bottom of the visible leaderboard, their rank doesn't fluctuate by exactly 127 places for every point gained - most rank changes should be minimal.

Whether you choose to disclose this estimated ranking as an estimation is entirely up to you. To add authenticity, consider incorporating a random element into the displayed number:

// Real rank: 22,838

// Displayed to user:
player rank: ~22.8k    // rounded
player rank: 22,882nd  // rounded with random salt of 44

In my case, I will be opting for the latter option.

Answer 4

Another creative solution I have come across involves estimating a user's rank in cases where they are not visible on the leaderboard. In my upcoming online game, I plan to implement this method which could also be applicable in your scenario. After all, does it really matter to the user whether they are ranked 22,882nd or 22,838th?

For example, if the player in 20th place has a score of 250 points and there are a total of 32,000 players, each point below 250 can represent an average drop of 127 places. However, implementing a curve may be beneficial so that as users approach the bottom of the visible leaderboard, their rank doesn't fluctuate by exactly 127 places for every point gained - most rank changes should be minimal.

Whether you choose to disclose this estimated ranking as an estimation is entirely up to you. To add authenticity, consider incorporating a random element into the displayed number:

// Real rank: 22,838

// Displayed to user:
player rank: ~22.8k    // rounded
player rank: 22,882nd  // rounded with random salt of 44

In my case, I will be opting for the latter option.

Answer 5

Answer №3

Looking at things from a different angle - opting for NoSQL and document stores can overcomplicate tasks like this. Postgres, on the other hand, makes it simple with its count function. While Firebase may seem convenient to start with, scenarios like this one really highlight the advantages of relational databases. If you're interested, check out Supabase at . It's similar to Firebase in terms of ease of use for setting up a backend quickly, but it's open source and utilizes Postgres as its foundation, providing the benefits of a relational database.

Answer 6

Looking at things from a different angle - opting for NoSQL and document stores can overcomplicate tasks like this. Postgres, on the other hand, makes it simple with its count function. While Firebase may seem convenient to start with, scenarios like this one really highlight the advantages of relational databases. If you're interested, check out Supabase at . It's similar to Firebase in terms of ease of use for setting up a backend quickly, but it's open source and utilizes Postgres as its foundation, providing the benefits of a relational database.

Answer 7

Answer №4

Another approach not mentioned by Dan involves utilizing security rules in conjunction with Google Cloud Functions.

Begin by setting up the highscores map as demonstrated below:

highScores (top20)

Next steps include:

Grant users write/read permissions for highScores.
Assign the smallest score to a property within the document/map highScores.
Allow users to only write new scores to highScores if their score is greater than the smallest score recorded.
Implement a write trigger within Google Cloud Functions to automatically remove the smallest score when a new highScore is added.

This method appears to be the most straightforward option and offers real-time functionality.

Answer 8

Another approach not mentioned by Dan involves utilizing security rules in conjunction with Google Cloud Functions.

Begin by setting up the highscores map as demonstrated below:

highScores (top20)

Next steps include:

Grant users write/read permissions for highScores.
Assign the smallest score to a property within the document/map highScores.
Allow users to only write new scores to highScores if their score is greater than the smallest score recorded.
Implement a write trigger within Google Cloud Functions to automatically remove the smallest score when a new highScore is added.

This method appears to be the most straightforward option and offers real-time functionality.

Answer 9

Answer №5

Firestore has recently introduced support for aggregation queries, including count(). Details can be found here.

For example, you can now execute:

const collectionRef = db.collection('cities');
const query = collectionRef.where('state', '==', 'CA');
const snapshot = await query.count().get();
console.log(snapshot.data().count);

To optimize costs, Google suggests combining the count operation with a where statement as "Pricing for count() depends on the number of index entries matched by the query".

Answer 10

Firestore has recently introduced support for aggregation queries, including count(). Details can be found here.

For example, you can now execute:

const collectionRef = db.collection('cities');
const query = collectionRef.where('state', '==', 'CA');
const snapshot = await query.count().get();
console.log(snapshot.data().count);

To optimize costs, Google suggests combining the count operation with a where statement as "Pricing for count() depends on the number of index entries matched by the query".

Answer 11

Answer №6

Utilizing cloud storage could be a solution in this scenario. One option is to create a file containing all users' scores in sequential order, allowing you to easily read the file and determine the position of a specific score.

To update the file, consider setting up a CRON job that periodically adds any new documents with a flag indicating they have not been written to the file yet. By aggregating these documents and marking them as processed, you can prevent excessive write operations. Retrieving a user's position by reading from the file should not be too resource-intensive, especially if implemented within a cloud function.

Answer 12

Utilizing cloud storage could be a solution in this scenario. One option is to create a file containing all users' scores in sequential order, allowing you to easily read the file and determine the position of a specific score.

To update the file, consider setting up a CRON job that periodically adds any new documents with a flag indicating they have not been written to the file yet. By aggregating these documents and marking them as processed, you can prevent excessive write operations. Retrieving a user's position by reading from the file should not be too resource-intensive, especially if implemented within a cloud function.

Answer 13

Answer №7

Revised in 2024 after receiving -2 downvotes:

Hello everyone, I wanted to provide an update as my original answer, while technically functional, is considered bad practice. Here's a better alternative:

In brief: utilize the count() aggregator in Firebase.

The count() aggregation is typically used for counting documents in collections or queries.

You can retrieve the count value (documents greater than the player position, for instance).

This allows you to determine the current player data position after sorting.

From there, you can calculate the top X players based on a specific measurement or target.

It's simple and effective.

Updated and Functional Response from 2022 (DO NOT UTILIZE MY PREVIOUS ANSWER)

To address the issue of creating leaderboards with user and points, and determining your position within this leaderboard more efficiently, I propose the following solution:

1) Create your Firestore Document structure as follows

https://i.sstatic.net/wq4U7.png

In my scenario, I have a document named perMission where each user has a field with the userId as the key and their respective leaderboard points as the value.

This setup makes it simpler to update values within my JavaScript code. For example, updating a user's points when they complete a mission:

import { doc, setDoc, increment } from "firebase/firestore"; 

const docRef = doc(db, 'leaderboards', 'perMission');
setDoc(docRef, { [userId]: increment(1) }, { merge: true });

You can customize the increment value as needed. In my case, I execute this code every time a user finishes a mission to increase their points.

2) Determining your position within the leaderboards

To ascertain your position on the client side, sort the values and iterate through them to find your place in the leaderboards.

While you can also use the object to access all users along with their respective points in order, my focus here is solely on determining my position.

The code is well-documented explaining each component:


// Values fetched from the database.
const leaderboards = {
  userId1: 1,
  userId2: 20,
  userId3: 30,
  userId4: 12,
  userId5: 40,
  userId6: 2
};

// Your user information.
const myUser = "userId4";
const myPoints = leaderboards[myUser];

// Sort the values in descending order.
const sortedLeaderboardsPoints = Object.values(leaderboards).sort(
  (a, b) => b - a
);

// Determine your specific position
const myPosition = sortedLeaderboardsPoints.reduce(
  (previous, current, index) => {
    if (myPoints === current) {
      return index + 1 + previous;
    }

    return previous;
  },
  0
);

// Output: [40, 30, 20, 12, 2, 1]
console.log(sortedLeaderboardsPoints);

// Output: 4
console.log(myPosition);

You can now leverage your position, even with a large array, since the logic operates on the client side. Exercise caution and consider optimizing the client-side code by reducing the array size or setting limits.

Remember to perform the remaining tasks on the client side rather than relying on Firebase alone.

This response aims to demonstrate proper database storage and usage practices.

Answer 14

Revised in 2024 after receiving -2 downvotes:

Hello everyone, I wanted to provide an update as my original answer, while technically functional, is considered bad practice. Here's a better alternative:

In brief: utilize the count() aggregator in Firebase.

The count() aggregation is typically used for counting documents in collections or queries.

You can retrieve the count value (documents greater than the player position, for instance).

This allows you to determine the current player data position after sorting.

From there, you can calculate the top X players based on a specific measurement or target.

It's simple and effective.

Updated and Functional Response from 2022 (DO NOT UTILIZE MY PREVIOUS ANSWER)

To address the issue of creating leaderboards with user and points, and determining your position within this leaderboard more efficiently, I propose the following solution:

1) Create your Firestore Document structure as follows

https://i.sstatic.net/wq4U7.png

In my scenario, I have a document named perMission where each user has a field with the userId as the key and their respective leaderboard points as the value.

This setup makes it simpler to update values within my JavaScript code. For example, updating a user's points when they complete a mission:

import { doc, setDoc, increment } from "firebase/firestore"; 

const docRef = doc(db, 'leaderboards', 'perMission');
setDoc(docRef, { [userId]: increment(1) }, { merge: true });

You can customize the increment value as needed. In my case, I execute this code every time a user finishes a mission to increase their points.

2) Determining your position within the leaderboards

To ascertain your position on the client side, sort the values and iterate through them to find your place in the leaderboards.

While you can also use the object to access all users along with their respective points in order, my focus here is solely on determining my position.

The code is well-documented explaining each component:


// Values fetched from the database.
const leaderboards = {
  userId1: 1,
  userId2: 20,
  userId3: 30,
  userId4: 12,
  userId5: 40,
  userId6: 2
};

// Your user information.
const myUser = "userId4";
const myPoints = leaderboards[myUser];

// Sort the values in descending order.
const sortedLeaderboardsPoints = Object.values(leaderboards).sort(
  (a, b) => b - a
);

// Determine your specific position
const myPosition = sortedLeaderboardsPoints.reduce(
  (previous, current, index) => {
    if (myPoints === current) {
      return index + 1 + previous;
    }

    return previous;
  },
  0
);

// Output: [40, 30, 20, 12, 2, 1]
console.log(sortedLeaderboardsPoints);

// Output: 4
console.log(myPosition);

You can now leverage your position, even with a large array, since the logic operates on the client side. Exercise caution and consider optimizing the client-side code by reducing the array size or setting limits.

Remember to perform the remaining tasks on the client side rather than relying on Firebase alone.

This response aims to demonstrate proper database storage and usage practices.

Firebase integration for tracking leaderboard positions

Answer №1

Simple Approach

Regular Update

Inverted Index

Sharded N-ary Tree

Final Thoughts

Answer №2

Answer №3

Answer №4

Answer №5

Answer №6

Answer №7

Revised in 2024 after receiving -2 downvotes:

Updated and Functional Response from 2022 (DO NOT UTILIZE MY PREVIOUS ANSWER)

1) Create your Firestore Document structure as follows

2) Determining your position within the leaderboards

Similar questions

The 'errorReason' property is not found within the 'MessageWithMdEnforced' type

Passing a variable from Twig to VueJS in Symfony 2.8 and VueJS 2: A beginner's guide

Leveraging a JavaScript variable within a PHP snippet

Is jest the ideal tool for testing an Angular Library?

Maximizing Performance: Enhancing Nested For Loops in Angular with Typescript

Tips for utilizing components in slots in Cypress and Vue for effective component testing

Turn off the div element until a radio button option is selected, then activate it

What should we name this particular style of navigation tab menu?

Is it possible for the Jquery Accordion to retract on click?

Working with Vue.js to call component methods when the page loads

Having trouble loading the linked CSS file in the Jade template

Can ng-submit be overridden?

What is the best way to merge my ajax request into a single line of code?

Tips for fixing a type error in javascript/cypress

Customize the popover in React Material-UI to your liking

Guidance on Implementing Promises in Ionic 2 and Angular 2

Obtain the index by clicking on an element within an HTMLCollection

Incorporate a dynamic PowerPoint presentation directly onto my website

JavaScript Filtering Techniques

For every iteration, verify the presence of the image