Maximizing performance through strategic database structuring with PouchDB/CouchDB

Recently, I've been researching efficient ways to store and retrieve data for my time management/project capturing application. I would greatly appreciate any insights on which storage strategy to utilize, or suggestions for alternative approaches, especially considering the limited local storage resources across different browsers.

Here's the main data I need to manage:

db_projects: This database stores the projects themselves.

db_timestamps: This contains timestamps for each project when it is active.


I've come up with the following strategies:

1: Storing project status in timestamps

When a project starts, a timestamp is added to db_timestamps like this:

db_timestamps.put({
   _id: String(Date.now()),
   title: projectID,
   status: status //could be: 1=active/2=inactive/3=paused
})...

This approach focuses on only adding data to the database without modifying entries. However, retrieving all active projects might require querying the entire db_timestamps which could be resource-intensive if it contains numerous entries.

2: Storing project status in db_projects

Whenever a project status changes, it is updated in the project itself. This makes querying for active projects more resource-friendly compared to timestamps. But, frequent status changes would lead to revisioning of project entries, potentially creating a lot of overhead. I'm unsure if the compaction feature would effectively handle this, as not all revision data is deleted entirely.

3: Storing status in a separate DB like db_status

This approach encounters similar challenges as #2, as status changes trigger revisions in this separate DB. If states are added in "only add data" mode, the DB could quickly become filled with entries.

Answer №1

The issue at hand revolves around the limited space available in indexedDB. Unlike CouchDB, which operates on the premise that storage space is cost-effective, especially when data is stored on the server side. For a fascinating discussion on this topic, check out this link.

Here is the approach I currently follow, which combines elements of both solution 1 and solution 2 mentioned earlier, along with some additional steps:

  1. Keep only timestamps in a synchronized database (db_timestamps) following the "only add data" principle.
  2. Store information about projects and their statuses in a separate local (unsynchronized) database (db_projects), utilizing PouchDB due to its simpler API compared to indexedDB.
  3. Include the new/modified project status in each timestamp, allowing for reconstruction of db_projects from db_timestamps if necessary.
  4. Periodically delete and repopulate db_projects to eliminate revision data overhead and maintain an acceptable size.

The following code snippet demonstrates how I reconstruct my database:

//--------------------------------------------------------------------
function rebuild_db_project(){   
    // Code for rebuilding database
}

//--------------------------------------------------------------------
function deleteDB(PouchDB_Name, IndexedDB_Name){
    // Code for deleting databases
}

In addition to using the pouchDB.destroy() command, I also leverage the indexedDB.deleteDatabase() command to free up storage space, although there might still be a small amount (around 4kB) that remains occupied but is inconsequential to me.

While the timing aspect may not be ideal, this setup currently serves my needs. Suggestions for improving the timing, particularly due to indexedDB's lack of promise support, would be greatly appreciated.

Similar questions

If you have not found the answer to your question or you are interested in this topic, then look at other similar questions below or use the search

Two entities positioned on opposite extremes

Is there a way to design a div with two elements (text and image) positioned on opposite sides? The length of the text varies as it's a list of months for selection. Ideally, the text should appear on the left side of the div as a "p" element, while t ...

What are the steps to integrate HJSON with Jest in a create-react-app development environment?

Currently, I am utilizing HJSON within a create-react-app project (view answer here). However, Jest does not utilize the same webpack configuration, resulting in a failed import of HJSON - for instance, import options from '../assets/options.hjson&ap ...

Tips for attaching a callback to Angular UI Popover's trigger

I recently implemented an Angular UI Popover in the following manner: <div popover-is-open="prfList.isProfileClosed===false" popover-trigger="'outsideClick'" popover-append-to-body="true" popover-placement="right-top" popover-class="popover1 ...

Breaking Down the Process of Exporting a Next.js Static Website into Manageable Parts

I am facing memory issues while building my website using Next.js's Static HTML Export. The site has a large number of static pages, approximately 10 million. Is it feasible to export these pages in batches, like exporting 100k pages in each build ite ...

Selecting a specific element and attaching a particular class to this element, but directing it towards a different element that shares the same index in a separate node list

I am working on a project where I have two sets of identical images - one is displayed in a gallery format and the other set is hidden until its corresponding gallery image is clicked, creating a lightbox effect. For example: <ul id="gallery"> ...

Encountering issues with extension CLI - "Unable to utilize the import statement outside a module"

Recently, I've been attempting to integrate the Afinn-165 node package (https://www.npmjs.com/package/afinn-165) into my Google Chrome extension. The goal is to analyze the sentiment of text scraped from each page. Being a novice in web development, I ...

My components views are not being rendered in Angular 4

Currently, I am in the process of learning how to use Angular 4, but I seem to be encountering an issue. Despite having a functioning App template that renders perfectly fine, I am facing difficulties when attempting to render more than one template. I cre ...

What exactly is the mechanism behind the functionality of ng-cloak?

Recently, I've been delving into the ng-cloak source code and trying to understand its inner workings. If you're interested, you can take a look at the source code here. From what I gather, it seems that the ng-cloak attribute is removed during ...

Navigating the world of gtag and google_tag_manager: untangling

Tracking custom events in my react application using Google Analytics has been successful. Initially, I followed a helpful document recommending the use of the gtag method over the ga method for logging calls. The implementation through Google Tag Manager ...

ReactJS component experiencing issues with loading images

In my React project, there is a directory containing several images, and I am attempting to import them all into a carousel using Bootstrap. The current code looks like this: import Carousel from 'react-bootstrap/Carousel'; const slideImagesFold ...

When attempting to log out of Keycloak, a TypeError occurs in the front-end application stating that it cannot read properties of undefined related to the action of logging out

I've been trying to implement a logout feature in my front-end application using Keycloak, but I'm encountering difficulties. Most of the examples I found online are for older versions of Keycloak and use 'auth' and 'redirectURI&ap ...

Implementing conditional visibility toggling in React

I am experiencing an issue where I am attempting to change a div's visibility from hidden to visible upon button click. Despite clicking the button, the visibility of the div does not change as expected. Upon inspecting the console after executing the ...

Creating a user-friendly form with validation in a Vue application using Vuetify.js

I am in the process of incorporating a contact form with basic validation on a Vue.js website using an example from Vuetify.js. Being new to this, I'm unsure about how to implement it within a Vue component. My goal is to have simple client-side form ...

Move two markers on Google Maps simultaneously

After researching various articles and posts on Stack Overflow about selecting multiple markers in Google Maps, I have been unable to find a solution for dragging two markers simultaneously. While I've managed to make it work so that one marker can tr ...

What is the procedure for obtaining a Connect server's host name and port number?

Just like the example shown in this Express-related question, I'm wondering if there is a way to programmatically retrieve the host name and port number of a running Connect server? ...

Ensuring the accurate promise is delivered in Angular

I'm struggling to correctly return the promise for a service in Angular. Here is the function causing me trouble: postToSP.post($scope.sharePointURL, data).then(function() { $scope.gettingData = false; $scope.yammerListName = ...

What is the best way to hide the next/previous tabs once the jQuery dataTable has been set up using jSON data

Setting up a jQuery table using JSON data. Despite knowing that there will only be one row, the next/previous tabs are still displayed after the table. Is there a way to remove them? Here is the code for the table: table = $("#retrievedTable").dataTabl ...

Issues with the directory for Chrome

Currently, I am utilizing jQuery and running an HTML file on my local machine without a server. Interestingly, the code works perfectly fine on Firefox but encounters issues on Chrome: $('#result').load('test.html'); It appears that t ...

What is the best way to streamline this using Javascript or jQuery?

Looking for a way to simplify the code for 4 vertical sliding panels? Check out the following script: $(".sliding-panels").click(function() { var n = $(this).attr("number"); var panels = $(".panel"); panels.not(".panel" + n).removeClass("pa ...

Is there a standardized method for obtaining a date in the format of six digits as YYMMDD?

In my current project, I'm developing a function that generates a list of dates represented in a 6-digit format beginning from the present day up until August of 2018. The desired output should resemble the following: [190322, 190321, 190320, ...] I ...