What causes the delay in monitoring the oplog in Meteor and Mongo?

I recently implemented Oplog tailing in my Meteor.js app using my MongoLab cluster to enhance performance, availability, and redundancy. However, I have noticed that since incorporating this feature, my publications are taking longer to complete. While a delay of around 200ms may not seem significant, in many cases it is taking much longer. This is particularly evident in a specific publication I have discussed here.

The response time for this publication is already lengthy, and the additional delay caused by oplog observations is further impacting it. This issue is not isolated to only one publication, as I have observed similar delays in other areas where oplog observations are involved.

I have searched extensively online for an explanation as to why observing oplogs is slowing down my publications, but have been unable to find any information. If anyone can shed light on this issue, I would greatly appreciate it.

Below are some Kadira screenshots that visually represent the performance disparities:

https://i.sstatic.net/UW5Zp.png

Here is a snapshot from another pub/sub:

https://i.sstatic.net/Lfeir.png

Lastly, an example where oplog observations have a reasonable impact on performance (though still affecting the pub/sub):

https://i.sstatic.net/dHU3E.png

javascriptmongodbmeteormongodb-oplog

Answer №1

Oplog tailing is indeed speedy. The problem does not lie in oplog tailing.

Chances are, there are several factors contributing to the sluggishness of your publications that you may not be aware of:

  • Iterating through each document and updating individually: Updating documents within a Collection.forEach loop can significantly slow down performance. Each update triggers updates for all concurrent connections, leading to a bottleneck. To optimize, consider using {multi:true} for updates.
  • Custom queries for each user: Making changes to a user document with multiple unique subscriptions can cause delays as notifications are sent sequentially. Reevaluate the necessity of individual subscriptions for each user and explore shared subscriptions for improved speed.
  • High volume of documents: Encoding each thread, comment, or chat message as a separate document can create overhead when sending to clients. Consider denormalizing data and consolidating information in a single document for improved efficiency.
  • Database not in proximity to host: If your database is not located in the same data center as your web host, latency issues may be impacting collection reads. While indices can help, the distance between your database and host may be a major factor in slow performance.

Similar questions

If you have not found the answer to your question or you are interested in this topic, then look at other similar questions below or use the search

The range slider's color value is not resetting correctly when using the <input>

Before repositioning, the slider is at this location: (both thumb and color are correctly positioned) https://i.stack.imgur.com/J3EGX.png Prior to Reset: Html <input id="xyzslider" class="mdl-slider mdl-js-slider" type="range" min="0.0" max="1.0" va ...

javascriptjqueryhtmlcssslider

Node.js - Synchronize asynchronous calls to ensure coordinated execution in code

I am trying to figure out how to make a for loop with an async function wait until all the async functions called within it are finished before allowing the code to continue. In my scenario, I have a variable "bar" that contains a JSON array with other ne ...

javascriptnode.jsloopsasynchronousnpm

How can I correctly export my npm package?

I have successfully developed an npm package. class Person { constructor() {} sayHello() {console.log("Hello World!")} } module.exports = Person; Upon installing the package using npm i my-package, it was integrated into my project. ...

javascriptnode.jsnpmnpm-publish

Expanding Java Classes and Replacing Methods with Multiple Parameters in ES4X/Graal

I am currently facing a challenge in my JavaScript project using ES4X/Graal, where I need to extend a Java class. This Java class has methods with overloaded parameters that I must override. While I understand how to call a specific Java method by specifyi ...

javascriptjavagraalvmgraaljs

What is the best way to choose documents in MongoDB where one field exceeds another?

Exploring my mongodb records, I came across this sample document format [_id] => MongoId Object ( [$id] => 52135a6baabacb9f948b4567 ) [change] => 7 [from] => 8 [to] => 1 I'm looking to filter documents where the value of "from" is ...

phpmongodb

What could be the reason for getting undefined when printing console.log(fibonacci(3))?

let array = [0, 1, 1]; const fibonacciSequence = (number) => { if (number < 2) { return array[number]; } else if (number == 2) { console.log(array.length-1); console.log((array[array.length-1])); // return (ar ...

javascript

Leveraging jQuery for invoking PHP functions

Seeking guidance on running mysql queries through JQuery. Any recommendations or tutorials available? I am working on integrating a system with jquery mobile, using a mysql database as the backend. Are there any resources out there to help me understand h ...

javascriptphpmysqlajaxjquery-mobile

The development server fails to respond when initializing a new project following the NextJs documentation for an empty project

After consulting the NextJs framework documentation, I meticulously followed the setup instructions to initialize an empty project : mkdir hello-next cd hello-next npm init -y npm install --save react react-dom next mkdir pages Subsequently, I included t ...

javascriptreactjsnext.js

Utilizing Vuetify 2 skeleton-loader to customize loading states through Vuex store manipulation

Utilizing the Vuetify v-skeleton-loader component to wrap a v-data-table component. The server-side pagination and sorting in the data-table component are set up. To enable server-side pagination, the documentation recommends monitoring the options objec ...

javascriptvue.jsvuejs2vuexvuetify.js

Utilize Material-UI's <Autocomplete /> feature to conduct searches using multiple parameters

Recently, I started delving into mastering Material UI and something has been on my mind. We are working with an array of objects as follows: const top100Films = [ { label: 'The Shawshank Redemption', year: 1994 }, { label: 'The Godfath ...

javascriptreactjsmaterial-ui

You can interact with our dropdown menu using the tab key, even when it is

We are looking to make our dropdown tabbable when expanded and non-tabbable when collapsed. We attempted using tabindex="-1" on the content inside the expandable div, but this resulted in it being non-tabbable even when expanded. Any suggestions on how t ...

javascripthtmlcssfrontend

A guide to adding a picture to AWS S3 with the help of GraphQL

When trying to upload a base64 string via GraphQL, I encountered an issue. It seems that if the string exceeds 50,000 characters, GraphQL fails to reach the resolve function without giving any error messages. However, when the string is less than 50,000 ...

javascriptreactjsamazon-s3graphqlaws-appsync

Running compiler-produced assembler code within the program

Recently, I came across an interesting presentation discussing inline ASM generation in Javascript optimization. The presentation can be found at the following link: . In another paper located here: https://sites.google.com/site/juliangamble/Home/Compiler ...

javascriptassemblyschemeinline-assemblydynamically-generated

Is there a way to determine if a button was clicked using twig?

I need assistance with implementing a button in my twig file within a table that will remove an element from an array. Ideally, I would like to remove the element based on its index. From the research I have conducted, it seems that data manipulation shou ...

javascriptphpjquerysymfonytwig

I'm attempting to retrieve information from my vuex store, however, encountering an error in the process

I've encountered an issue with vuex getters while working on my project. I have a route that showcases all users, and upon visiting this route, the AllUsers.vue component is displayed. Within this component, I'm utilizing the UsersList.vue compo ...

javascriptvue.jsvuexvuejs3website-deployment

Utilize JQuery variables for toggling the visibility of various DIV elements

On my webpage's splash page, there are 4 divs but only the home div is initially visible. The other three are hidden. Each of these divs has a button associated with it that triggers a jquery click event to swap out the currently visible div for the ...

javascriptjqueryhtmlcss

Deletion of component with setTimeout in React Class Component

I have a notification feature that disappears after a certain delay when rendered. The issue arises when attempting to cancel this automatic removal using clearTimeout, as it doesn't seem to work. See below class Notify extends React.Component { ...

javascriptreactjs

Support for time-series data in MongoDB through ingestion from Debezium

My server receives data from multiple devices generated by their sensors. This telemetry data is stored in json format in a partitioned table named statuses in a postgresql database. The table is partitioned monthly on the columns time_stamp and device_id, ...

mongodbpostgresqldebezium

Display unformatted JSON code and format it into a visually appealing structure within a DIV element

I am currently working on a function that accepts unformatted JSON code as input and I am attempting to utilize the stringify method to format it into a more visually appealing (pretty) view. Here is the code snippet: function prettyPrintJSON(selectionInf ...

javascriptjsonformatpretty-print

How does Python interpret arrays from JavaScript?

As part of my API setup, I am sending parameters to a python script. One of these parameters happens to be a JavaScript array. Interestingly, when I check the array in Python, it only shows the first index. Here is the snippet of my Angular JS get request ...

javascriptarrays